{"thread":{"id":"11871","subject":"git-fetch in 1.5.4 fails versus 1.5.3.8","startedAt":"2008-02-04T18:25:25Z","lastAt":"2008-02-09T21:51:19Z","messageCount":44,"participants":["Anand Kumria","Jeff King","Jari Aalto","Mike Hommey","Frank Lichtenheld","Linus Torvalds","Martin Langhoff","Dmitry Potapov","Junio C Hamano","Johannes Schindelin","Florian Weimer","Daniel Stenberg"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"67396","messageId":"pan.2008.02.04.18.25.26@progsoc.org","threadId":"11871","inReplyTo":null,"subject":"git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Anand Kumria","fromEmail":"wildfire@progsoc.org","sentAt":"2008-02-04T18:25:25Z","receivedAt":"2008-02-04T18:25:25Z","isPatch":false,"sender":{"key":"wildfire@progsoc.org","avatar":null},"body":"\nHi,\n\nHaving recently upgraded to git-core 1.5.4, I seen to have stumbled onto \na regression.\n\n$ git --version\ngit version 1.5.4\n\n$ cat .git/config\n{{ ... }}\n[remote \"richard\"]\n url = https://server.example.com/~richard/newfoo.git\n fetch = +refs/heads/*:refs/remotes/richard/*\n\n$ git fetch richard\nerror: (curl_result = 77, http_code = 0, sha1 = \n0bc27e5162d0e74053b71fc637cbbf8fc942e969)\nGetting pack list for https://server.example.com/~richard/newfoo.git\nerror:\nGetting alternates list for https://server.example.com/~richard/newfoo.git\nerror: Unable to find 0bc27e5162d0e74053b71fc637cbbf8fc942e969 under \nhttps://server.example.com/~richard/newfoo.git\nCannot obtain needed object 0bc27e5162d0e74053b71fc637cbbf8fc942e969\nfatal: Fetch failed.\n\nBut:\n\n\n$ git clone https://server.example.com/~richard/newfoo.git\nInitialized empty Git repository in /home/anand/projects/newfoo/.git/\nGetting alternates list for https://server.example.com/~richard/newfoo.git\nGetting pack list for https://server.example.com/~richard/newfoo.git\nGetting index for pack 4b4ae2516826a864230a1e2e83e3cf900e7dbbb3\nGetting index for pack 28255a0fb8b9369afe9b46b79164de160c0c532d\nGetting index for pack a43ad1dff20585c7f903f498c65260a26cf57a3c\nGetting index for pack 78e01c912dc4987099c40a68cbb741cb69365522\nGetting index for pack ee32d85b391fda784dd5afccb22434746f112acc\nGetting pack a43ad1dff20585c7f903f498c65260a26cf57a3c\n which contains 12daf1b07314589a93c2e3dbe7cb0a2f3074f4af\nwalk 12daf1b07314589a93c2e3dbe7cb0a2f3074f4af\nwalk 15542942acf5021eb911ee80a8c89f7c2bdb471e\nwalk cf3558da086dccc24c76c371917df73c0cfd1b6f\nGetting pack 4b4ae2516826a864230a1e2e83e3cf900e7dbbb3\n which contains c761504e5e08c9111a819a4a707d86f860a24afa\nGetting pack 28255a0fb8b9369afe9b46b79164de160c0c532d\n which contains df0eb371252791a066a3ebdd7feeb445245fcb80\nwalk 1496f6f7ffc39f44a1dc26584baf68c4b62ebfb5\n[snip]\nwalk bce69a02c5ea897a4a1302cb603a74d9f19afa9f\nwalk 7bec5801836ee2b2486093da74deee0b39e693c3\ngot 0bc27e5162d0e74053b71fc637cbbf8fc942e969\nwalk 0bc27e5162d0e74053b71fc637cbbf8fc942e969\n[snip]\nGetting alternates list for https://server.example.com/~richard/newfoo.git\ngot e7ddd78769bf781707c2fed5e6b9c3c8d827b89b\nwalk e326801f90a554bc0af089c8a7afffa45662fd7d\nGetting pack list for https://server.example.com/~richard/newfoo.git\nGetting pack 78e01c912dc4987099c40a68cbb741cb69365522\n which contains 0da3b81806aa51d854f06c4a61fe45dafbdc66d3\n[snip\ngot ac162a07d389e110aa6725e4ef2a3eedc42f05bd\ngot 227139e3eef336917f8a50aba06cd5e172608899\n\n\nDowngrading to git-core in Debian (1.5.3.8) and it works perfectly.\n\n$ git fetch richard\nFetching refs/heads/master from https://server.example.com/~richard/\nnewfoo.git using https\nFetching refs/heads/master-richard from https://server.example.com/\n~richard/newfoo.git using https\ngot 0bc27e5162d0e74053b71fc637cbbf8fc942e969\nwalk 0bc27e5162d0e74053b71fc637cbbf8fc942e969\ngot bb7bfc531acee412ea945928073fadef5eba0fb4\ngot 1223925015090bad2b4b4e3cc0524a23a9bd644c\ngot 785e2d3d4f9834cf9c5c81d89d590c82d82c032c\n* refs/remotes/richard/master-richard: fast forward to branch 'master-\nrichard' of https://server.example.com/~richard/newfoo\n old..new: e326801..0bc27e5\nFetching refs/heads/resellers from https://server.example.com/~richard/\nnewfoo.git using https\n\nSuggestions?\n\nAnand\n"},{"id":"67482","messageId":"20080205050741.GA4624@coredump.intra.peff.net","threadId":"11871","inReplyTo":"pan.2008.02.04.18.25.26@progsoc.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-05T05:07:41Z","receivedAt":"2008-02-05T05:07:41Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 04, 2008 at 06:25:25PM +0000, Anand Kumria wrote:\n\n> $ cat .git/config\n> {{ ... }}\n> [remote \"richard\"]\n> url = https://server.example.com/~richard/newfoo.git\n> fetch = +refs/heads/*:refs/remotes/richard/*\n> \n> $ git fetch richard\n> error: (curl_result = 77, http_code = 0, sha1 = \n> 0bc27e5162d0e74053b71fc637cbbf8fc942e969)\n\nI was unable to reproduce your problem here. However, peeking in curl's\nheader files, it looks like error 77 is:\n\n CURLE_SSL_CACERT_BADFILE, /* 77 - could not load CACERT file, missing\n or wrong format */\n\n> Downgrading to git-core in Debian (1.5.3.8) and it works perfectly.\n\nSince you are running Debian, can you confirm whether you have the\n'ca-certificates' package installed? It creates the\n/etc/ssl/certs/ca-certificates.crt file, which is presumably the source\nof the complaining.\n\nThat being said, there seems to be some difference between 1.5.3.8 and\n1.5.4 that made us care more about SSL certs (though I note that the\nSSL_VERIFYPEER curl knob has been set since pre-1.0). Have you tried\nsetting GIT_SSL_NO_VERIFY?\n\n-Peff\n"},{"id":"67521","messageId":"zlufxx9b.fsf@blue.sea.net","threadId":"11871","inReplyTo":"20080205050741.GA4624@coredump.intra.peff.net","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jari Aalto","fromEmail":"jari.aalto@cante.net","sentAt":"2008-02-05T15:01:52Z","receivedAt":"2008-02-05T15:01:52Z","isPatch":false,"sender":{"key":"jari.aalto@cante.net","avatar":"https://avatars.githubusercontent.com/u/34601?v=4"},"body":"* Tue 2008-02-05 Jeff King gmane.comp.version-control.git\n* Message-Id: 20080205050741.GA4624@coredump.intra.peff.net\n> On Mon, Feb 04, 2008 at 06:25:25PM +0000, Anand Kumria wrote:\n> That being said, there seems to be some difference between 1.5.3.8 and\n> 1.5.4 that made us care more about SSL certs (though I note that the\n> SSL_VERIFYPEER curl knob has been set since pre-1.0). Have you tried\n> setting GIT_SSL_NO_VERIFY?\n\nConfirmed. The \"git push\" returned failure, but when compiled with this\noptions, it works ok.\n\n $ uname -a\n SunOS 5.9 Generic_118558-35 sun4u sparc SUNW,Serverblade1\n\nJari\n\n-- \nWelcome to FOSS revolution: we fix and modify until it shines\n"},{"id":"67715","messageId":"pan.2008.02.06.21.56.35@progsoc.org","threadId":"11871","inReplyTo":"20080205050741.GA4624@coredump.intra.peff.net","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Anand Kumria","fromEmail":"wildfire@progsoc.org","sentAt":"2008-02-06T21:56:35Z","receivedAt":"2008-02-06T21:56:35Z","isPatch":false,"sender":{"key":"wildfire@progsoc.org","avatar":null},"body":"On Tue, 05 Feb 2008 00:07:41 -0500, Jeff King wrote:\n\n> On Mon, Feb 04, 2008 at 06:25:25PM +0000, Anand Kumria wrote:\n> \n>> $ cat .git/config\n>> {{ ... }}\n>> [remote \"richard\"]\n>> url = https://server.example.com/~richard/newfoo.git fetch =\n>> +refs/heads/*:refs/remotes/richard/*\n>> \n>> $ git fetch richard\n>> error: (curl_result = 77, http_code = 0, sha1 =\n>> 0bc27e5162d0e74053b71fc637cbbf8fc942e969)\n> \n> I was unable to reproduce your problem here. However, peeking in curl's\n\nI bisected this down to the patch that makes git-fetch a builtin \n(b888d61c8308027433df9c243fa551f42db1c76a) -- which is where I guessed \nthe functionality changed.\n\n> header files, it looks like error 77 is:\n> \n> CURLE_SSL_CACERT_BADFILE, /* 77 - could not load CACERT file,\n> missing\n> or wrong format */\n> \n>> Downgrading to git-core in Debian (1.5.3.8) and it works perfectly.\n> \n> Since you are running Debian, can you confirm whether you have the\n> 'ca-certificates' package installed? \n\nYes, I have it installed.\n\n> It creates the\n> /etc/ssl/certs/ca-certificates.crt file, which is presumably the source\n> of the complaining.\n\nYes, I have this file.\n\n\n> That being said, there seems to be some difference between 1.5.3.8 and\n> 1.5.4 that made us care more about SSL certs (though I note that the\n> SSL_VERIFYPEER curl knob has been set since pre-1.0). Have you tried\n> setting GIT_SSL_NO_VERIFY?\n> \n> -Peff\n\nWith git 1.5.4 I can merrily clone the repository via https without issue.\n\nOnly using git-fetch seems to have an issue.\n\nWith GIT_SSL_NO_VERIFY defined, it fails with:\n\n$ GIT_SSL_NO_VERIFY=1 ../git/git-fetch richard\nerror: gnutls_handshake() failed: ASN1 parser: Element was not found. (curl_result = 35, http_code = 0, sha1 = 510567ca41e201253445528ca6eb89ed43e71fce)\nGetting pack list for https://server.example.com/~richard/newfoo.git\nerror: gnutls_handshake() failed: ASN1 parser: Element was not found.\nGetting alternates list for https://server.example.com/~richard/newfoo.git\nerror: Unable to find 510567ca41e201253445528ca6eb89ed43e71fce under https://server.example.com/~richard/newfoo.git\nCannot obtain needed object 510567ca41e201253445528ca6eb89ed43e71fce\nfatal: Fetch failed.\n\n\nI *think* what is happening is that it is it is trying for the object - not seeing it and then not attempting to get the pack file.\n\nBut I'm having a hard time debugging this as git-fetch launches git-rev-list internally and it seems to be failing in there, really.\n\nThanks,\nAnand\n\nBisect log:\n\ngit-bisect start\n# good: [aadd4efa715f56e0eac5ac459c8ff4933b56d4ce] GIT 1.5.3.8\ngit-bisect good aadd4efa715f56e0eac5ac459c8ff4933b56d4ce\n# bad: [c3c135291a62a01f7fd385f46cde34091767259b] GIT 1.5.4\ngit-bisect bad c3c135291a62a01f7fd385f46cde34091767259b\n# bad: [183f84365de7b4b1fe0e15cebce80a95023aa1d6] git-p4: Fix typo in --detect-labels\ngit-bisect bad 183f84365de7b4b1fe0e15cebce80a95023aa1d6\n# bad: [6ca8b977e4f678050db8fcb0eec2091dd44a2bd0] Bisect: add \"skip\" to the short usage string.\ngit-bisect bad 6ca8b977e4f678050db8fcb0eec2091dd44a2bd0\n# good: [e66273a6abb8e9cd0967d52113e29c8014a255f8] Merge branch 'lh/merge'\ngit-bisect good e66273a6abb8e9cd0967d52113e29c8014a255f8\n# good: [f5bf6feb05b8c89c448ded6e6fad0eb58ef35463] Merge branch 'maint'\ngit-bisect good f5bf6feb05b8c89c448ded6e6fad0eb58ef35463\n# bad: [2b5a06edca8f7237aad6464b349b79772024d2a2] Restore default verbosity for http fetches.\ngit-bisect bad 2b5a06edca8f7237aad6464b349b79772024d2a2\n# bad: [3278cd0a39c30c6c3082fc5feed0f9bd98b5f628] Properly cleanup in http_cleanup so builtin-fetch does not segfault\ngit-bisect bad 3278cd0a39c30c6c3082fc5feed0f9bd98b5f628\n# good: [c7a8a16239c6bdbb4041dd8a8773ae055d3cccf8] Add bundle transport\ngit-bisect good c7a8a16239c6bdbb4041dd8a8773ae055d3cccf8\n# bad: [7a2bff45937a60d846abf3ccb42015539aedcb40] Replace custom memory growth allocator with ALLOC_GROW\ngit-bisect bad 7a2bff45937a60d846abf3ccb42015539aedcb40\n# bad: [4ad1eada9774a1f340beb4fdf78f1735534741bb] Fix off by one bug in reflog messages written by builtin-fetch\ngit-bisect bad 4ad1eada9774a1f340beb4fdf78f1735534741bb\n# bad: [1aad91f5a715af92892aea7764beb829938ab111] Correct builtin-fetch to handle + in refspecs\ngit-bisect bad 1aad91f5a715af92892aea7764beb829938ab111\n# bad: [b888d61c8308027433df9c243fa551f42db1c76a] Make fetch a builtin\ngit-bisect bad b888d61c8308027433df9c243fa551f42db1c76a\n"},{"id":"67747","messageId":"20080207042332.GA7632@sigill.intra.peff.net","threadId":"11871","inReplyTo":"pan.2008.02.06.21.56.35@progsoc.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-07T04:23:33Z","receivedAt":"2008-02-07T04:23:33Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 06, 2008 at 09:56:35PM +0000, Anand Kumria wrote:\n\n> With GIT_SSL_NO_VERIFY defined, it fails with:\n> \n> $ GIT_SSL_NO_VERIFY=1 ../git/git-fetch richard\n> error: gnutls_handshake() failed: ASN1 parser: Element was not found. (curl_result = 35, http_code = 0, sha1 = 510567ca41e201253445528ca6eb89ed43e71fce)\n> Getting pack list for https://server.example.com/~richard/newfoo.git\n> error: gnutls_handshake() failed: ASN1 parser: Element was not found.\n> Getting alternates list for https://server.example.com/~richard/newfoo.git\n> error: Unable to find 510567ca41e201253445528ca6eb89ed43e71fce under https://server.example.com/~richard/newfoo.git\n> Cannot obtain needed object 510567ca41e201253445528ca6eb89ed43e71fce\n> fatal: Fetch failed.\n\nOK, I was finally able to reproduce your bug. It seems that it _only_\nhappens when using curl built against gnutls. I built against the\nlibcurl4-openssl-dev in Debian unstable, and the problem goes away.\n\nCan you confirm that building using the openssl version of curl fixes\nthe problem?\n\nGoogling for your error message turns up only one other instance: a bug\nin pidgin where the result was \"this seems like a bug in gnutls.\" I hate\nto say \"it's not our bug\" without knowing exactly what is causing it,\nthough. And it does seem odd that it works with 1.5.3.8. I wonder if\nthere is some difference in the way we are calling curl that matters.\n\n> I *think* what is happening is that it is it is trying for the object\n> - not seeing it and then not attempting to get the pack file.\n\nNo, it only fails to see objects because of the curl failure (it tries\nthe loose and then the pack).\n\n-Peff\n"},{"id":"67757","messageId":"20080207063714.GB19561@glandium.org","threadId":"11871","inReplyTo":"20080207042332.GA7632@sigill.intra.peff.net","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-07T06:37:14Z","receivedAt":"2008-02-07T06:37:14Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Wed, Feb 06, 2008 at 11:23:33PM -0500, Jeff King wrote:\n> On Wed, Feb 06, 2008 at 09:56:35PM +0000, Anand Kumria wrote:\n> \n> > With GIT_SSL_NO_VERIFY defined, it fails with:\n> > \n> > $ GIT_SSL_NO_VERIFY=1 ../git/git-fetch richard\n> > error: gnutls_handshake() failed: ASN1 parser: Element was not found. (curl_result = 35, http_code = 0, sha1 = 510567ca41e201253445528ca6eb89ed43e71fce)\n> > Getting pack list for https://server.example.com/~richard/newfoo.git\n> > error: gnutls_handshake() failed: ASN1 parser: Element was not found.\n> > Getting alternates list for https://server.example.com/~richard/newfoo.git\n> > error: Unable to find 510567ca41e201253445528ca6eb89ed43e71fce under https://server.example.com/~richard/newfoo.git\n> > Cannot obtain needed object 510567ca41e201253445528ca6eb89ed43e71fce\n> > fatal: Fetch failed.\n> \n> OK, I was finally able to reproduce your bug. It seems that it _only_\n> happens when using curl built against gnutls. I built against the\n> libcurl4-openssl-dev in Debian unstable, and the problem goes away.\n> \n> Can you confirm that building using the openssl version of curl fixes\n> the problem?\n> \n> Googling for your error message turns up only one other instance: a bug\n> in pidgin where the result was \"this seems like a bug in gnutls.\" I hate\n> to say \"it's not our bug\" without knowing exactly what is causing it,\n> though. And it does seem odd that it works with 1.5.3.8. I wonder if\n> there is some difference in the way we are calling curl that matters.\n\nNothing significant.\n\nMike\n"},{"id":"67778","messageId":"pan.2008.02.07.10.15.05@progsoc.org","threadId":"11871","inReplyTo":"20080207042332.GA7632@sigill.intra.peff.net","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Anand Kumria","fromEmail":"wildfire@progsoc.org","sentAt":"2008-02-07T10:15:02Z","receivedAt":"2008-02-07T10:15:02Z","isPatch":false,"sender":{"key":"wildfire@progsoc.org","avatar":null},"body":"On Wed, 06 Feb 2008 23:23:33 -0500, Jeff King wrote:\n\n> On Wed, Feb 06, 2008 at 09:56:35PM +0000, Anand Kumria wrote:\n> \n>> With GIT_SSL_NO_VERIFY defined, it fails with:\n>> \n>> $ GIT_SSL_NO_VERIFY=1 ../git/git-fetch richard error:\n>> gnutls_handshake() failed: ASN1 parser: Element was not found.\n>> (curl_result = 35, http_code = 0, sha1 =\n>> 510567ca41e201253445528ca6eb89ed43e71fce) Getting pack list for\n>> https://server.example.com/~richard/newfoo.git error:\n>> gnutls_handshake() failed: ASN1 parser: Element was not found. Getting\n>> alternates list for https://server.example.com/~richard/newfoo.git\n>> error: Unable to find 510567ca41e201253445528ca6eb89ed43e71fce under\n>> https://server.example.com/~richard/newfoo.git Cannot obtain needed\n>> object 510567ca41e201253445528ca6eb89ed43e71fce fatal: Fetch failed.\n> \n> OK, I was finally able to reproduce your bug. It seems that it _only_\n> happens when using curl built against gnutls. I built against the\n> libcurl4-openssl-dev in Debian unstable, and the problem goes away.\n> \n> Can you confirm that building using the openssl version of curl fixes\n> the problem?\n\nConfirmed.\n\nThanks for figuring out how to reproduce it ... how did you btw?\n\n> Googling for your error message turns up only one other instance: a bug\n> in pidgin where the result was \"this seems like a bug in gnutls.\" I hate\n> to say \"it's not our bug\" without knowing exactly what is causing it,\n> though. And it does seem odd that it works with 1.5.3.8. I wonder if\n> there is some difference in the way we are calling curl that matters.\n\nIt appears that git 1.5.3.8 on Debian links to libcurl3-gnutls whereas, \nat least for me, git 1.5.4 on Debian links to libcurl4-gnutls \n(or libcurl4-openssl).\n\nI agree with you, it is a bit problematic when the library (curl) relies\non another library (gnutls) and the bottom one is having a problem.\n\nGerrit - since I seem to be able to reproduce this fairly easily - would\nit be useful to you to have me do anything to track this down. Or will you\nswitch the Debian build to openssl?\n\nThanks for looking into this Jeff.\n\nRegards,\nAnand\n"},{"id":"67780","messageId":"20080207110601.GA8488@coredump.intra.peff.net","threadId":"11871","inReplyTo":"pan.2008.02.07.10.15.05@progsoc.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-07T11:06:02Z","receivedAt":"2008-02-07T11:06:02Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 07, 2008 at 10:15:02AM +0000, Anand Kumria wrote:\n\n> > OK, I was finally able to reproduce your bug. It seems that it _only_\n> > happens when using curl built against gnutls. I built against the\n> > libcurl4-openssl-dev in Debian unstable, and the problem goes away.\n> \n> Thanks for figuring out how to reproduce it ... how did you btw?\n\nI saw the gnutls error message in your output and took a guess that it\nwas related. I was able to reproduce against the first https repository\nthat I tried (I don't think it has anything to do with the repository).\n\nI wish we could more certainly blame it on something besides git,\nthough. I can't reproduce it using just 'curl', so it's possible that\nthere is a problem with the way git is calling libcurl.\n\n> It appears that git 1.5.3.8 on Debian links to libcurl3-gnutls whereas, \n> at least for me, git 1.5.4 on Debian links to libcurl4-gnutls \n> (or libcurl4-openssl).\n> \n> I agree with you, it is a bit problematic when the library (curl) relies\n> on another library (gnutls) and the bottom one is having a problem.\n\nIt would be nice if we could generate a minimal test case that\ndemonstrates the problem, but I can't seem to reproduce it with a\nsmaller program. If we could, then we could probably get advice from\ncurl and/or gnutls people.\n\n-Peff\n"},{"id":"67781","messageId":"20080207121042.GA10210@glandium.org","threadId":"11871","inReplyTo":"20080207110601.GA8488@coredump.intra.peff.net","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-07T12:10:43Z","receivedAt":"2008-02-07T12:10:43Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Thu, Feb 07, 2008 at 06:06:02AM -0500, Jeff King wrote:\n> On Thu, Feb 07, 2008 at 10:15:02AM +0000, Anand Kumria wrote:\n> \n> > > OK, I was finally able to reproduce your bug. It seems that it _only_\n> > > happens when using curl built against gnutls. I built against the\n> > > libcurl4-openssl-dev in Debian unstable, and the problem goes away.\n> > \n> > Thanks for figuring out how to reproduce it ... how did you btw?\n> \n> I saw the gnutls error message in your output and took a guess that it\n> was related. I was able to reproduce against the first https repository\n> that I tried (I don't think it has anything to do with the repository).\n> \n> I wish we could more certainly blame it on something besides git,\n> though. I can't reproduce it using just 'curl', so it's possible that\n> there is a problem with the way git is calling libcurl.\n> \n> > It appears that git 1.5.3.8 on Debian links to libcurl3-gnutls whereas, \n> > at least for me, git 1.5.4 on Debian links to libcurl4-gnutls \n> > (or libcurl4-openssl).\n> > \n> > I agree with you, it is a bit problematic when the library (curl) relies\n> > on another library (gnutls) and the bottom one is having a problem.\n> \n> It would be nice if we could generate a minimal test case that\n> demonstrates the problem, but I can't seem to reproduce it with a\n> smaller program. If we could, then we could probably get advice from\n> curl and/or gnutls people.\n\nDid you try to run with the GIT_SSL_NO_VERIFY environment variable set ?\n\nMike\n"},{"id":"67783","messageId":"20080207122842.GA17184@coredump.intra.peff.net","threadId":"11871","inReplyTo":"20080207121042.GA10210@glandium.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-07T12:28:42Z","receivedAt":"2008-02-07T12:28:42Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 07, 2008 at 01:10:43PM +0100, Mike Hommey wrote:\n\n> > It would be nice if we could generate a minimal test case that\n> > demonstrates the problem, but I can't seem to reproduce it with a\n> > smaller program. If we could, then we could probably get advice from\n> > curl and/or gnutls people.\n> \n> Did you try to run with the GIT_SSL_NO_VERIFY environment variable set ?\n\nYes (I even suggested this earlier in the thread). It returns a\ndifferent error from gnutls (see Anand's earlier response).\n\n-Peff\n"},{"id":"67805","messageId":"20080207142322.GC18497@mail-vs.djpig.de","threadId":"11871","inReplyTo":"pan.2008.02.07.10.15.05@progsoc.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Frank Lichtenheld","fromEmail":"frank@lichtenheld.de","sentAt":"2008-02-07T14:23:22Z","receivedAt":"2008-02-07T14:23:22Z","isPatch":false,"sender":{"key":"frank@lichtenheld.de","avatar":"https://gravatar.com/avatar/b9f1d4b120e138f157c9e480d0818197c474628923786adb98f30017cdb99c3c?d=mp&s=160"},"body":"On Thu, Feb 07, 2008 at 10:15:02AM +0000, Anand Kumria wrote:\n> Gerrit - since I seem to be able to reproduce this fairly easily - would\n> it be useful to you to have me do anything to track this down. Or will you\n> switch the Debian build to openssl?\n\nSince git has no OpenSSL link exception the resulting binary wouldn't be\ndistributable AFAIK.\n\nGruesse,\n-- \nFrank Lichtenheld \nwww: http://www.djpig.de/\n"},{"id":"67842","messageId":"alpine.LFD.1.00.0802071039010.2883@woody.linux-foundation.org","threadId":"11871","inReplyTo":"20080207142322.GC18497@mail-vs.djpig.de","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-02-07T18:42:15Z","receivedAt":"2008-02-07T18:42:15Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 7 Feb 2008, Frank Lichtenheld wrote:\n> \n> Since git has no OpenSSL link exception the resulting binary wouldn't be\n> distributable AFAIK.\n\nFor crazy people who think that regular libraries can change the copyright \nstatus of a program (not so), you can always decide to build without \nOpenSSL and use the included Mozilla-based SHA1 implementation for git.\n\nPerformance will probably suffer, and maybe something else breaks too (I \ndoubt many people test the build that way very often), but I assume Debian \npeople don't care.\n\nAfter all, if you're a Debian person, it's likely more important to you to \nbe difficult and anal and argue about theoretical license details than \nactually be *usable*.\n\n\t\tLinus\n"},{"id":"67853","messageId":"20080207201445.GD18497@mail-vs.djpig.de","threadId":"11871","inReplyTo":"alpine.LFD.1.00.0802071039010.2883@woody.linux-foundation.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Frank Lichtenheld","fromEmail":"frank@lichtenheld.de","sentAt":"2008-02-07T20:14:45Z","receivedAt":"2008-02-07T20:14:45Z","isPatch":false,"sender":{"key":"frank@lichtenheld.de","avatar":"https://gravatar.com/avatar/b9f1d4b120e138f157c9e480d0818197c474628923786adb98f30017cdb99c3c?d=mp&s=160"},"body":"On Thu, Feb 07, 2008 at 10:42:15AM -0800, Linus Torvalds wrote:\n> On Thu, 7 Feb 2008, Frank Lichtenheld wrote:\n> > \n> > Since git has no OpenSSL link exception the resulting binary wouldn't be\n> > distributable AFAIK.\n> \n> For crazy people who think that regular libraries can change the copyright \n> status of a program (not so), you can always decide to build without \n> OpenSSL and use the included Mozilla-based SHA1 implementation for git.\n> \n> Performance will probably suffer, and maybe something else breaks too (I \n> doubt many people test the build that way very often), but I assume Debian \n> people don't care.\n> \n> After all, if you're a Debian person, it's likely more important to you to \n> be difficult and anal and argue about theoretical license details than \n> actually be *usable*.\n\nEasy to say for someone who only distributes source code... (AFAIK\nanyway)\n\nAnyway, since Debian will not change its opinion about this, my answer\nwas in the context of the question obviously useful. Whether it was\ngenerally correct is probably off-topic here.\n\nGruesse,\n-- \nFrank Lichtenheld \nwww: http://www.djpig.de/\n"},{"id":"67855","messageId":"20080207204026.GA2550@sigill.intra.peff.net","threadId":"11871","inReplyTo":"alpine.LFD.1.00.0802071039010.2883@woody.linux-foundation.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-07T20:40:29Z","receivedAt":"2008-02-07T20:40:29Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 07, 2008 at 10:42:15AM -0800, Linus Torvalds wrote:\n\n> For crazy people who think that regular libraries can change the copyright \n> status of a program (not so), you can always decide to build without \n> OpenSSL and use the included Mozilla-based SHA1 implementation for git.\n\nFWIW, this is not about OpenSSL for SHA1; it is about the underlying\nlibrary used by curl to do SSL (gnutls vs openssl). And the problem is\nthat curl linked against gnutls seems _broken_, so Anand has asked if\nDebian can ship a binary git linked against a curl that is linked\nagainst openssl (and the answer is probably \"no, Debian people think\nthat is wrong\").\n\n-Peff\n"},{"id":"67856","messageId":"alpine.LFD.1.00.0802071246320.2896@woody.linux-foundation.org","threadId":"11871","inReplyTo":"20080207201445.GD18497@mail-vs.djpig.de","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-02-07T20:54:30Z","receivedAt":"2008-02-07T20:54:30Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 7 Feb 2008, Frank Lichtenheld wrote:\n> \n> Easy to say for someone who only distributes source code... (AFAIK\n> anyway)\n\nSure. But I'd like to point out that there are tons of distributiors of \nLinux _and_ other operating systems - with real lawyers involved - that \ndistribute things compiled with OpenSSL, and nobody sane actually thinks \nit is a problem. The fact that the OpenSSL license isn't compatible with \nGPL is a total non-issue: people compile GPL'd programs against totally \nproprietary libraries or other non-GPL-compatible things.\n\n> Anyway, since Debian will not change its opinion about this, my answer\n> was in the context of the question obviously useful. Whether it was\n> generally correct is probably off-topic here.\n\nUmm. You claimed that the result would not be \"distributable\". I just \nboth corrected that total misunderstanding (on part of the Debian crowd) \n_and_ said that even crazy Debian people can work around it.\n\nSo please don't say that things are not \"distributable\" when they clearly \nare. \n\n\t\t\tLinus\n"},{"id":"67859","messageId":"alpine.LFD.1.00.0802071256570.2896@woody.linux-foundation.org","threadId":"11871","inReplyTo":"20080207204026.GA2550@sigill.intra.peff.net","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-02-07T21:01:09Z","receivedAt":"2008-02-07T21:01:09Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 7 Feb 2008, Jeff King wrote:\n> \n> FWIW, this is not about OpenSSL for SHA1; it is about the underlying\n> library used by curl to do SSL (gnutls vs openssl).\n\nMy comment was about claiming \"not distributable\". That was simply not \ntrue. It's perfectly distributable, it's just Debian that has issues with \nOpenSSL (but then they shouldn't link it against curl either, so there \nseems to be some _other_ problem there too).\n\n> And the problem is that curl linked against gnutls seems _broken_, so \n> Anand has asked if Debian can ship a binary git linked against a curl \n> that is linked against openssl (and the answer is probably \"no, Debian \n> people think that is wrong\").\n\nSure. And you can probably fix it by using NO_OPENSSL, which uses the \nMozilla SHA1 library. As I also pointed out.\n\nIn short - I just wanted to make sure that we do not make the insane \nDebian policies somehow official git ones.\n\n\t\t\tLinus\n"},{"id":"67861","messageId":"971f65790802071336k2383b408k66b3cebfb4e1da13@mail.gmail.com","threadId":"11871","inReplyTo":"alpine.LFD.1.00.0802071246320.2896@woody.linux-foundation.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Anand Kumria","fromEmail":"wildfire@progsoc.org","sentAt":"2008-02-07T21:36:32Z","receivedAt":"2008-02-07T21:36:32Z","isPatch":false,"sender":{"key":"wildfire@progsoc.org","avatar":null},"body":"On 2/7/08, Linus Torvalds wrote:\n>\n>\n> On Thu, 7 Feb 2008, Frank Lichtenheld wrote:\n> >\n> > Easy to say for someone who only distributes source code... (AFAIK\n> > anyway)\n>\n> Sure. But I'd like to point out that there are tons of distributiors of\n> Linux _and_ other operating systems - with real lawyers involved - that\n> distribute things compiled with OpenSSL, and nobody sane actually thinks\n> it is a problem.\n\nWhoa.\n\nI didn't mean to start a flamefest about distributable or not within\nDebian. (disclaimer, I'm a Debian developer).\n\nIf there is anything I can do to debug the issue with gnutls - let me\nknow. I'm just stumped about how to debug the problem.\n\nAnand\n"},{"id":"67863","messageId":"46a038f90802071347l2e6465a1v85e4f5a21b96a109@mail.gmail.com","threadId":"11871","inReplyTo":"alpine.LFD.1.00.0802071256570.2896@woody.linux-foundation.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2008-02-07T21:47:00Z","receivedAt":"2008-02-07T21:47:00Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Feb 8, 2008 10:01 AM, Linus Torvalds wrote:\n> My comment was about claiming \"not distributable\". That was simply not\n> true. It's perfectly distributable, it's just Debian that has issues with\n> OpenSSL (but then they shouldn't link it against curl either, so there\n> seems to be some _other_ problem there too).\n\nI think tides are shifting and Debian may be seeing some common sense\nprevail. Thanks to ubuntu perhaps, or solar flares...\n\n> In short - I just wanted to make sure that we do not make the insane\n> Debian policies somehow official git ones.\n\nGood clarification anyway :-)\n\n\nm\n"},{"id":"67865","messageId":"20080207215309.GP30368@dpotapov.dyndns.org","threadId":"11871","inReplyTo":"alpine.LFD.1.00.0802071256570.2896@woody.linux-foundation.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-02-07T21:53:09Z","receivedAt":"2008-02-07T21:53:09Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Thu, Feb 07, 2008 at 01:01:09PM -0800, Linus Torvalds wrote:\n> \n> On Thu, 7 Feb 2008, Jeff King wrote:\n> > \n> > FWIW, this is not about OpenSSL for SHA1; it is about the underlying\n> > library used by curl to do SSL (gnutls vs openssl).\n> \n> My comment was about claiming \"not distributable\". That was simply not \n> true. It's perfectly distributable, it's just Debian that has issues with \n> OpenSSL (but then they shouldn't link it against curl either, so there \n> seems to be some _other_ problem there too).\n\nThe curl license is very permissive, so there is no problem to link it\nagainst any GPL program. OTOH, the OpenSSL is more restrictive than GPL,\nand because GPL is copyleft (i.e. it prevents adding any restriction on\nany derived work) to distribute Git linked against OpenSSL is technically\nillegal unless OpenSSL is the part of the standard OS libraries or Git\ndevelopers provide a special exemption that allows to link Git against\nOpenSSL and to redistribute the result. For more details, see\nhttp://www.gnome.org/~markmc/openssl-and-the-gpl.html\n\nDmitry\n"},{"id":"67867","messageId":"20080207220243.GA7237@glandium.org","threadId":"11871","inReplyTo":"20080207122842.GA17184@coredump.intra.peff.net","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-07T22:02:43Z","receivedAt":"2008-02-07T22:02:43Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Thu, Feb 07, 2008 at 07:28:42AM -0500, Jeff King wrote:\n> On Thu, Feb 07, 2008 at 01:10:43PM +0100, Mike Hommey wrote:\n> \n> > > It would be nice if we could generate a minimal test case that\n> > > demonstrates the problem, but I can't seem to reproduce it with a\n> > > smaller program. If we could, then we could probably get advice from\n> > > curl and/or gnutls people.\n> > \n> > Did you try to run with the GIT_SSL_NO_VERIFY environment variable set ?\n> \n> Yes (I even suggested this earlier in the thread). It returns a\n> different error from gnutls (see Anand's earlier response).\n\nSorry, I've had trouble opening my eyes and actually reading messages I\nreply to... anyways, I tried to reproduce with curl-gnutls and...\ncouldn't... How did you manage that ? Is the server you were trying on\npublic ? Do you have any http.ssl* variables set in your configuration ?\n\nMike\n"},{"id":"67869","messageId":"20080207224054.GA18502@coredump.intra.peff.net","threadId":"11871","inReplyTo":"alpine.LFD.1.00.0802071256570.2896@woody.linux-foundation.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-07T22:40:54Z","receivedAt":"2008-02-07T22:40:54Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 07, 2008 at 01:01:09PM -0800, Linus Torvalds wrote:\n\n> > FWIW, this is not about OpenSSL for SHA1; it is about the underlying\n> > library used by curl to do SSL (gnutls vs openssl).\n> \n> My comment was about claiming \"not distributable\". That was simply not \n> true. It's perfectly distributable, it's just Debian that has issues with \n> OpenSSL (but then they shouldn't link it against curl either, so there \n> seems to be some _other_ problem there too).\n\nAnd I happen to agree with you, but...\n\n> > And the problem is that curl linked against gnutls seems _broken_, so \n> > Anand has asked if Debian can ship a binary git linked against a curl \n> > that is linked against openssl (and the answer is probably \"no, Debian \n> > people think that is wrong\").\n> \n> Sure. And you can probably fix it by using NO_OPENSSL, which uses the \n> Mozilla SHA1 library. As I also pointed out.\n\nwhat I was saying before is that NO_OPENSSL _doesn't_ fix the problem,\nbecause this has nothing whatsoever to do with the mozilla sha1 library\nor any decision that git can make.\n\nDebian provides two versions of curl, one that uses openssl and one that\nuses gnutls. The question of which is used depends on which Debian\npackage you happen to have installed. So it is not a git matter at all,\nbut rather a matter of Debian policy about which version of curl is used\nwhen building the official binary packages.\n\n> In short - I just wanted to make sure that we do not make the insane \n> Debian policies somehow official git ones.\n\nAgreed. There is no fallout from this issue for git; it is purely a\nDebian build process issue.\n\n-Peff\n"},{"id":"67870","messageId":"20080207224640.GB18502@coredump.intra.peff.net","threadId":"11871","inReplyTo":"20080207215309.GP30368@dpotapov.dyndns.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-07T22:46:40Z","receivedAt":"2008-02-07T22:46:40Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Feb 08, 2008 at 12:53:09AM +0300, Dmitry Potapov wrote:\n\n> any derived work) to distribute Git linked against OpenSSL is technically\n> illegal unless OpenSSL is the part of the standard OS libraries or Git\n\nFWIW, openssl is in the \"important\" section in Debian, which makes it a\npart of the standard install.\n\nBut again, this doesn't matter at all for git itself, but only for\nbinary distributors who must follow whatever interpretation of the GPL\ntheir distribution decrees.\n\n-Peff\n"},{"id":"67874","messageId":"20080207232337.GR30368@dpotapov.dyndns.org","threadId":"11871","inReplyTo":"pan.2008.02.07.10.15.05@progsoc.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-02-07T23:23:37Z","receivedAt":"2008-02-07T23:23:37Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Thu, Feb 07, 2008 at 10:15:02AM +0000, Anand Kumria wrote:\n> On Wed, 06 Feb 2008 23:23:33 -0500, Jeff King wrote:\n> \n> > Googling for your error message turns up only one other instance: a bug\n> > in pidgin where the result was \"this seems like a bug in gnutls.\" I hate\n> > to say \"it's not our bug\" without knowing exactly what is causing it,\n> > though. And it does seem odd that it works with 1.5.3.8. I wonder if\n> > there is some difference in the way we are calling curl that matters.\n> \n> It appears that git 1.5.3.8 on Debian links to libcurl3-gnutls whereas, \n> at least for me, git 1.5.4 on Debian links to libcurl4-gnutls \n> (or libcurl4-openssl).\n\nHave you tried Git 1.5.4 with libcurl3-gnutls? It seems the package from\nDebian unstable is built with it. I have backported Git 1.5.4 to Etch with\nlibcurl3-gnutls and I have not noticed any problems with https fetch. So,\nI wonder if the problem happens with libcurl4-gnutls only regardless of\nyour version of Git.\n\nDmitry\n"},{"id":"67880","messageId":"20080208003239.GA18856@coredump.intra.peff.net","threadId":"11871","inReplyTo":"20080207220243.GA7237@glandium.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-08T00:32:39Z","receivedAt":"2008-02-08T00:32:39Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 07, 2008 at 11:02:43PM +0100, Mike Hommey wrote:\n\n> Sorry, I've had trouble opening my eyes and actually reading messages I\n> reply to... anyways, I tried to reproduce with curl-gnutls and...\n> couldn't... How did you manage that ? Is the server you were trying on\n> public ? Do you have any http.ssl* variables set in your configuration ?\n\nNo, my test repo is not public. I have no special ssl configuration\n(though I do use GIT_SSL_NO_VERIFY=1 since I just had a test self-signed\ncert). The exact recipe on my Debian system is:\n\n1. Build broken git on client\n\n client$ apt-get install libcurl4-gnutls-dev\n client$ cd ~/compile/git && make install\n\n2. On server with ssl-enabled webserver (I am using lighttpd, but\n I don't think that it matters), make a small repo.\n\n server$ cd /var/www\n server$ mkdir foo && cd foo && git init\n server$ echo one >file && git add file && git commit -m one\n server$ git update-server-info\n\n3. On client, clone repo, which should work ok\n\n client$ export GIT_SSL_NO_VERIFY=1 ;# if necessary\n client$ git clone https://yourserver.com/foo/.git\n\n4. make a new commit in parent repo\n\n server$ echo two >file && git commit -a -m two\n server$ git update-server-info\n\n5. fetch from client (this output is with GIT_SSL_NO_VERIFY=1; you get\n slightly different output if verification is on)\n\n client$ git fetch\n error: gnutls_handshake() failed: ASN1 parser: Element was not found.\n (curl_result = 35, http_code = 0, sha1 =\n 07ac7bd2edd32a5818d719145910119ab72c9dd4)\n Getting pack list for https://peff.net/git/foo/.git\n error: gnutls_handshake() failed: ASN1 parser: Element was not found.\n Getting alternates list for https://peff.net/git/foo/.git\n error: Unable to find 07ac7bd2edd32a5818d719145910119ab72c9dd4 under\n https://peff.net/git/foo/.git\n Cannot obtain needed object 07ac7bd2edd32a5818d719145910119ab72c9dd4\n fatal: Fetch failed.\n\n6. On client, rebuild with libcurl4-openssl-dev\n\n client$ apt-get install libcurl4-openssl-dev\n client$ cd ~/compile/git && make clean install\n\n7. On client, do the fetch, which now works\n\n client$ cd ~/foo && git fetch\n\n-Peff\n"},{"id":"67884","messageId":"pan.2008.02.08.02.43.21@progsoc.org","threadId":"11871","inReplyTo":"20080207232337.GR30368@dpotapov.dyndns.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Anand Kumria","fromEmail":"wildfire@progsoc.org","sentAt":"2008-02-08T02:43:19Z","receivedAt":"2008-02-08T02:43:19Z","isPatch":false,"sender":{"key":"wildfire@progsoc.org","avatar":null},"body":"On Fri, 08 Feb 2008 02:23:37 +0300, Dmitry Potapov wrote:\n\n> On Thu, Feb 07, 2008 at 10:15:02AM +0000, Anand Kumria wrote:\n>> On Wed, 06 Feb 2008 23:23:33 -0500, Jeff King wrote:\n>> \n>> > Googling for your error message turns up only one other instance: a\n>> > bug in pidgin where the result was \"this seems like a bug in gnutls.\"\n>> > I hate to say \"it's not our bug\" without knowing exactly what is\n>> > causing it, though. And it does seem odd that it works with 1.5.3.8.\n>> > I wonder if there is some difference in the way we are calling curl\n>> > that matters.\n>> \n>> It appears that git 1.5.3.8 on Debian links to libcurl3-gnutls whereas,\n>> at least for me, git 1.5.4 on Debian links to libcurl4-gnutls (or\n>> libcurl4-openssl).\n> \n> Have you tried Git 1.5.4 with libcurl3-gnutls? It seems the package from\n> Debian unstable is built with it. I have backported Git 1.5.4 to Etch\n> with libcurl3-gnutls and I have not noticed any problems with https\n\nYes. I've tried the Debian git 1.5.3.8 and git 1.5.4 with whatever they \nare linked to (libcurl3-gnutls as you point out).\n\nWhen I decided to build & bisect to see if I could troubleshoot, I ended \nup building with libcurl4-gnutls-dev installed first. When compiled \nagainst libcurl4-openssl-dev things works. \n\nSo it definately seems specific to how git uses libcurl and how it, in \nturn, uses gnutls.\n\nAnand\n"},{"id":"67903","messageId":"20080208071835.GA11807@glandium.org","threadId":"11871","inReplyTo":"20080208003239.GA18856@coredump.intra.peff.net","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T07:18:35Z","receivedAt":"2008-02-08T07:18:35Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Thu, Feb 07, 2008 at 07:32:39PM -0500, Jeff King wrote:\n> On Thu, Feb 07, 2008 at 11:02:43PM +0100, Mike Hommey wrote:\n> \n> > Sorry, I've had trouble opening my eyes and actually reading messages I\n> > reply to... anyways, I tried to reproduce with curl-gnutls and...\n> > couldn't... How did you manage that ? Is the server you were trying on\n> > public ? Do you have any http.ssl* variables set in your configuration ?\n> \n> No, my test repo is not public. I have no special ssl configuration\n> (though I do use GIT_SSL_NO_VERIFY=1 since I just had a test self-signed\n> cert). The exact recipe on my Debian system is:\n> \n(...)\n\nOkay, I've been able to reproduce the problem. I don't know what I've\nbeen doing wrong to have it hidden...\n\nAnyways, the interesting thing is to look at what curl has to say in its\nverbose mode:\n\nGIT_CURL_VERBOSE=1 git fetch\n* Couldn't find host localhost in the .netrc file, using defaults\n* About to connect() to localhost port 8443 (#0)\n* Trying 127.0.0.1... * connected\n* Connected to localhost (127.0.0.1) port 8443 (#0)\n* found 102 certificates in /etc/ssl/certs/ca-certificates.crt\n* server certificate verification FAILED\n* common name: localhost (matched)\n* server certificate expiration date OK\n* server certificate activation date OK\n* certificate public key: RSA\n* certificate version: #1\n* subject: C=GB,ST=Some-State,L=Some-Locality,O=One Organization,OU=One Organization Unit,CN=localhost,EMAIL=webmaster@localhost\n* start date: Thu, 07 Feb 2008 21:27:36 GMT\n* expire date: Sat, 08 Mar 2008 21:27:36 GMT\n* issuer: C=GB,ST=Some-State,L=Some-Locality,O=One Organization,OU=One Organization Unit,CN=localhost,EMAIL=webmaster@localhost\n* compression: DEFLATE\n* cipher: AES 256 CBC\n* MAC: SHA\n> GET /foo/.git//info/refs HTTP/1.1\nUser-Agent: git/1.5.4.7.gd8534-dirty\nHost: localhost:8443\nAccept: */*\n\n< HTTP/1.1 200 OK\n< Date: Fri, 08 Feb 2008 07:10:09 GMT\n< Server: Apache/2.2.8 (Debian) DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g\n< Last-Modified: Fri, 08 Feb 2008 06:52:19 GMT\n< ETag: \"61d82e-3b-445a0080d0ec0\"\n< Accept-Ranges: bytes\n< Content-Length: 59\n< Content-Type: text/plain\n< \n* Connection #0 to host localhost left intact\n* Couldn't find host localhost in the .netrc file, using defaults\n* About to connect() to localhost port 8443 (#0)\n* Trying 127.0.0.1... * connected\n* Connected to localhost (127.0.0.1) port 8443 (#0)\n* error reading ca cert file /etc/ssl/certs/ca-certificates.crt (ASN1 parser: Element was not found.)\n* gnutls_handshake() failed: ASN1 parser: Element was not found.\n* Expire cleared\n* Closing connection #0\nerror: gnutls_handshake() failed: ASN1 parser: Element was not found. (curl_result = 35, http_code = 0, sha1 = e0aa43ffb1a1e7052a936b9ed5e0a1462cfc343e)\nGetting pack list for https://localhost:8443/foo/.git\n\nSo, it looks like either gnutls or curl is doing something wrong and\ncan't parse /etc/ssl/certs/ca-certificates.crt a second time. This\nlooks like a bug in either curl or gnutls.\n\nA simplified testcase would probably be to do two requests in a row, but\nI don't have time right now to do this testing.\n\nMike\n"},{"id":"67906","messageId":"20080208073456.GA17791@glandium.org","threadId":"11871","inReplyTo":"20080208071835.GA11807@glandium.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T07:34:56Z","receivedAt":"2008-02-08T07:34:56Z","isPatch":false,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Fri, Feb 08, 2008 at 08:18:35AM +0100, Mike Hommey wrote:\n> On Thu, Feb 07, 2008 at 07:32:39PM -0500, Jeff King wrote:\n> > On Thu, Feb 07, 2008 at 11:02:43PM +0100, Mike Hommey wrote:\n> > \n> > > Sorry, I've had trouble opening my eyes and actually reading messages I\n> > > reply to... anyways, I tried to reproduce with curl-gnutls and...\n> > > couldn't... How did you manage that ? Is the server you were trying on\n> > > public ? Do you have any http.ssl* variables set in your configuration ?\n> > \n> > No, my test repo is not public. I have no special ssl configuration\n> > (though I do use GIT_SSL_NO_VERIFY=1 since I just had a test self-signed\n> > cert). The exact recipe on my Debian system is:\n> > \n> (...)\n> \n> Okay, I've been able to reproduce the problem. I don't know what I've\n> been doing wrong to have it hidden...\n> \n> Anyways, the interesting thing is to look at what curl has to say in its\n> verbose mode:\n(...)\n> \n> So, it looks like either gnutls or curl is doing something wrong and\n> can't parse /etc/ssl/certs/ca-certificates.crt a second time. This\n> looks like a bug in either curl or gnutls.\n> \n> A simplified testcase would probably be to do two requests in a row, but\n> I don't have time right now to do this testing.\n\nI'm making myself a liar, but I took some few more minutes to test\nsomething like:\ndiff --git a/http.c b/http.c\nindex d2c11ae..001b1c4 100644\n--- a/http.c\n+++ b/http.c\n@@ -186,7 +186,7 @@ static CURL* get_curl_handle(void)\n if (ssl_capath != NULL)\n curl_easy_setopt(result, CURLOPT_CAPATH, ssl_capath);\n #endif\n- if (ssl_cainfo != NULL)\n+// if (ssl_cainfo != NULL)\n curl_easy_setopt(result, CURLOPT_CAINFO, ssl_cainfo);\n curl_easy_setopt(result, CURLOPT_FAILONERROR, 1);\n \n\n\nAnd the result is interesting:\nGIT_CURL_VERBOSE=1 git fetch\n* Couldn't find host localhost in the .netrc file, using defaults\n* About to connect() to localhost port 8443 (#0)\n* Trying 127.0.0.1... * connected\n* Connected to localhost (127.0.0.1) port 8443 (#0)\n* server certificate verification FAILED\n* common name: localhost (matched)\n* server certificate expiration date OK\n* server certificate activation date OK\n* certificate public key: RSA\n* certificate version: #1\n* subject: C=GB,ST=Some-State,L=Some-Locality,O=One Organization,OU=One Organization Unit,CN=localhost,EMAIL=webmaster@localhost\n* start date: Thu, 07 Feb 2008 21:27:36 GMT\n* expire date: Sat, 08 Mar 2008 21:27:36 GMT\n* issuer: C=GB,ST=Some-State,L=Some-Locality,O=One Organization,OU=One Organization Unit,CN=localhost,EMAIL=webmaster@localhost\n* compression: DEFLATE\n* cipher: AES 256 CBC\n* MAC: SHA\n> GET /foo/.git//info/refs HTTP/1.1\nUser-Agent: git/1.5.4.7.gd8534-dirty\nHost: localhost:8443\nAccept: */*\n\n< HTTP/1.1 200 OK\n< Date: Fri, 08 Feb 2008 07:30:10 GMT\n< Server: Apache/2.2.8 (Debian) DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g\n< Last-Modified: Fri, 08 Feb 2008 06:52:19 GMT\n< ETag: \"61d82e-3b-445a0080d0ec0\"\n< Accept-Ranges: bytes\n< Content-Length: 59\n< Content-Type: text/plain\n< \n* Connection #0 to host localhost left intact\n* Couldn't find host localhost in the .netrc file, using defaults\n* About to connect() to localhost port 8443 (#0)\n* Trying 127.0.0.1... * connected\n* Connected to localhost (127.0.0.1) port 8443 (#0)\n* gnutls_handshake() failed: ASN1 parser: Element was not found.\n* Expire cleared\n* Closing connection #0\nerror: gnutls_handshake() failed: ASN1 parser: Element was not found. (curl_result = 35, http_code = 0, sha1 = e0aa43ffb1a1e7052a936b9ed5e0a1462cfc343e)\n\nSo, it looks like either gnutls has a problem reinitializing its ASN1\nparser or curl is doing something wrong with gnutls when initializing a\nnew request.\n\nMike\n"},{"id":"67943","messageId":"20080208132721.GW30368@dpotapov.dyndns.org","threadId":"11871","inReplyTo":"pan.2008.02.08.02.43.21@progsoc.org","subject":"Re: git-fetch in 1.5.4 fails versus 1.5.3.8","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-02-08T13:27:21Z","receivedAt":"2008-02-08T13:27:21Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Fri, Feb 08, 2008 at 02:43:19AM +0000, Anand Kumria wrote:\n> \n> Yes. I've tried the Debian git 1.5.3.8 and git 1.5.4 with whatever they \n> are linked to (libcurl3-gnutls as you point out).\n> \n> When I decided to build & bisect to see if I could troubleshoot, I ended \n> up building with libcurl4-gnutls-dev installed first. When compiled \n> against libcurl4-openssl-dev things works. \n> \n> So it definately seems specific to how git uses libcurl and how it, in \n> turn, uses gnutls.\n\nI have investigated this issue a bit more... As I mentioned before I\nused Git 1.5.4 with libcurl3-gnutls on Debian Etch and did not have\nthat problem, but when I installed exactly the same package on Debian\nunstable it exhibits the above problem. Debian Etch has curl v7.15.5,\nwhile Debian testing uses curl v7.17.1 and Debian unstable uses curl\n7.18.0 (both libcurl3 and libcurl4 are built from the same sources).\nSo the version of libcurl seems to be relevant here. OTOH, git 1.5.3.8\nworks with both versions of libcurl-gnutls, while git 1.5.4 does not\nwork with a new one. So, it is also specific to how git uses libcurl.\nI will look into it more during the weekend.\n\nDmitry\n"},{"id":"67985","messageId":"1202501335-28205-1-git-send-email-mh@glandium.org","threadId":"11871","inReplyTo":"20080208073456.GA17791@glandium.org","subject":"[PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T20:08:55Z","receivedAt":"2008-02-08T20:08:55Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"Either recent curl or gnutls doesn't like initializing again after cleaning\nup, and this is happening in some cases such as git fetch.\n\nWe work around this by removing the http_cleanup call from get_refs_via_curl,\nand allowing http_init to be called several times without initializing http.c\nglobal variables again and leaking old values.\n\nThe remaining calls to http_cleanup are either last (http-push.c), or almost\nnever called (walker.c; the function it lies in is only called from\ntransport-disconnect, which is called last, and only in builtin-push.c).\nThese leaks shall be addressed in the http code refactoring.\n\nSigned-off-by: Mike Hommey \n---\n\n > So, it looks like either gnutls has a problem reinitializing its ASN1\n > parser or curl is doing something wrong with gnutls when initializing a\n > new request.\n\n In the end, it was a bit of git's fault, but either curl or gnutls is the\n actual culprit. I've not looked into either code to find out who's\n responsible, but a very simplified testcase is as follows:\n\n\t#include \n\t#include \n\n\tint main(void) {\n\t CURL *easy = curl_easy_init();\n\t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n\t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n\t curl_easy_perform(easy);\n\t curl_global_cleanup();\n\t easy = curl_easy_init();\n\t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n\t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n\t curl_easy_perform(easy);\n\t}\n\n\t(build with gcc -o test test.c -lcurl)\n\t(note curl_easy_init does curl_global_init behind the curtains, even the\n\tsecond time. You can convince yourself by adding\n\tcurl_global_init(CURL_GLOBAL_ALL);)\n\n http.c | 5 +++++\n transport.c | 2 --\n 2 files changed, 5 insertions(+), 2 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex d2c11ae..a3aa9e9 100644\n--- a/http.c\n+++ b/http.c\n@@ -215,9 +215,14 @@ static CURL* get_curl_handle(void)\n \n void http_init(void)\n {\n+\tstatic int init = 0;\n \tchar *low_speed_limit;\n \tchar *low_speed_time;\n \n+\tif (init)\n+\t\treturn;\n+\tinit = 1;\n+\n \tcurl_global_init(CURL_GLOBAL_ALL);\n \n \tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\ndiff --git a/transport.c b/transport.c\nindex babaa21..1eb6d78 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -473,8 +473,6 @@ static struct ref *get_refs_via_curl(struct transport *transport)\n \t\treturn NULL;\n \t}\n \n-\thttp_cleanup();\n-\n \tdata = buffer.buf;\n \tstart = NULL;\n \tmid = data;\n-- \n1.5.4.7.gd8534-dirty\n"},{"id":"68008","messageId":"20080208213148.GA2823@glandium.org","threadId":"11871","inReplyTo":"1202501335-28205-1-git-send-email-mh@glandium.org","subject":"Re: [PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T21:31:48Z","receivedAt":"2008-02-08T21:31:48Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"> In the end, it was a bit of git's fault, but either curl or gnutls is the\n> actual culprit. I've not looked into either code to find out who's\n> responsible, but a very simplified testcase is as follows:\n> \n> \t#include \n> \t#include \n> \n> \tint main(void) {\n> \t CURL *easy = curl_easy_init();\n> \t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n> \t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n> \t curl_easy_perform(easy);\n> \t curl_global_cleanup();\n> \t easy = curl_easy_init();\n> \t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n> \t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n> \t curl_easy_perform(easy);\n> \t}\n> \n> \t(build with gcc -o test test.c -lcurl)\n> \t(note curl_easy_init does curl_global_init behind the curtains, even the\n> \tsecond time. You can convince yourself by adding\n> \tcurl_global_init(CURL_GLOBAL_ALL);)\n\nAnd the winner is... curl !\nThe bug was introduced in this commit:\nhttp://cool.haxx.se/cvs.cgi/curl/lib/gtls.c.diff?r1=1.26&r2=1.27\nNote how gtls_inited is not set back to FALSE in cleanup.\n\nThis ended up released in 7.16.3. I'm filing a bug.\n\nMike\n"},{"id":"68011","messageId":"7vlk5vi0k9.fsf@gitster.siamese.dyndns.org","threadId":"11871","inReplyTo":"20080208213148.GA2823@glandium.org","subject":"Re: [PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-08T21:46:14Z","receivedAt":"2008-02-08T21:46:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Mike Hommey writes:\n\n>> In the end, it was a bit of git's fault, but either curl or gnutls is the\n>> actual culprit. I've not looked into either code to find out who's\n>> responsible, but a very simplified testcase is as follows:\n>> ...\n>\n> And the winner is... curl !\n> The bug was introduced in this commit:\n> http://cool.haxx.se/cvs.cgi/curl/lib/gtls.c.diff?r1=1.26&r2=1.27\n> Note how gtls_inited is not set back to FALSE in cleanup.\n>\n> This ended up released in 7.16.3. I'm filing a bug.\n\nGood detetive work. Thanks.\n\nI guess we need to ship with a known leak to work this around.\nSigh...\n\nPerhaps we can convince cURL developers to switch to git while\nwe are at it? ;-)\n"},{"id":"68013","messageId":"20080208215140.GA21362@glandium.org","threadId":"11871","inReplyTo":"7vlk5vi0k9.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T21:51:40Z","receivedAt":"2008-02-08T21:51:40Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Fri, Feb 08, 2008 at 01:46:14PM -0800, Junio C Hamano wrote:\n> Mike Hommey writes:\n> \n> >> In the end, it was a bit of git's fault, but either curl or gnutls is the\n> >> actual culprit. I've not looked into either code to find out who's\n> >> responsible, but a very simplified testcase is as follows:\n> >> ...\n> >\n> > And the winner is... curl !\n> > The bug was introduced in this commit:\n> > http://cool.haxx.se/cvs.cgi/curl/lib/gtls.c.diff?r1=1.26&r2=1.27\n> > Note how gtls_inited is not set back to FALSE in cleanup.\n> >\n> > This ended up released in 7.16.3. I'm filing a bug.\n> \n> Good detetive work. Thanks.\n> \n> I guess we need to ship with a known leak to work this around.\n> Sigh...\n\nWe can probably add a test on curl versions to avoid leaking on every\ninstall. Something like #if LIBCURL_VERSION_NUM < 0x071003. And then add\n|| LIBCURL_VERSION_NUM > .... whenever this is fixed in curl...\nThough, as I said, we are not calling http_cleanup in a lot of cases,\nalready.\n\nMike\n"},{"id":"68015","messageId":"alpine.LSU.1.00.0802082152410.11591@racer.site","threadId":"11871","inReplyTo":"20080208213148.GA2823@glandium.org","subject":"Re: [PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-02-08T21:53:47Z","receivedAt":"2008-02-08T21:53:47Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 8 Feb 2008, Mike Hommey wrote:\n\n> > In the end, it was a bit of git's fault, but either curl or gnutls is \n> > the actual culprit. I've not looked into either code to find out \n> > who's responsible, but a very simplified testcase is as follows:\n> > \n> > \t#include \n> > \t#include \n> > \n> > \tint main(void) {\n> > \t CURL *easy = curl_easy_init();\n> > \t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n> > \t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n> > \t curl_easy_perform(easy);\n> > \t curl_global_cleanup();\n> > \t easy = curl_easy_init();\n> > \t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n> > \t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n> > \t curl_easy_perform(easy);\n> > \t}\n> > \n> > \t(build with gcc -o test test.c -lcurl)\n> > \t(note curl_easy_init does curl_global_init behind the curtains, \n> > even the second time. You can convince yourself by adding\n> > \t curl_global_init(CURL_GLOBAL_ALL);)\n> \n> And the winner is... curl !\n> The bug was introduced in this commit:\n> http://cool.haxx.se/cvs.cgi/curl/lib/gtls.c.diff?r1=1.26&r2=1.27\n> Note how gtls_inited is not set back to FALSE in cleanup.\n\nWow. I hope you used \"git bisect\", in order to spare you unnecessary \nwork...\n\n> This ended up released in 7.16.3. I'm filing a bug.\n\nThanks for being so persistent.\n\nCiao,\nDscho\n"},{"id":"68016","messageId":"20080208220123.GA21882@glandium.org","threadId":"11871","inReplyTo":"alpine.LSU.1.00.0802082152410.11591@racer.site","subject":"Re: [PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T22:01:23Z","receivedAt":"2008-02-08T22:01:23Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Fri, Feb 08, 2008 at 09:53:47PM +0000, Johannes Schindelin wrote:\n> Hi,\n> \n> On Fri, 8 Feb 2008, Mike Hommey wrote:\n> \n> > > In the end, it was a bit of git's fault, but either curl or gnutls is \n> > > the actual culprit. I've not looked into either code to find out \n> > > who's responsible, but a very simplified testcase is as follows:\n> > > \n> > > \t#include \n> > > \t#include \n> > > \n> > > \tint main(void) {\n> > > \t CURL *easy = curl_easy_init();\n> > > \t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n> > > \t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n> > > \t curl_easy_perform(easy);\n> > > \t curl_global_cleanup();\n> > > \t easy = curl_easy_init();\n> > > \t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n> > > \t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n> > > \t curl_easy_perform(easy);\n> > > \t}\n> > > \n> > > \t(build with gcc -o test test.c -lcurl)\n> > > \t(note curl_easy_init does curl_global_init behind the curtains, \n> > > even the second time. You can convince yourself by adding\n> > > \t curl_global_init(CURL_GLOBAL_ALL);)\n> > \n> > And the winner is... curl !\n> > The bug was introduced in this commit:\n> > http://cool.haxx.se/cvs.cgi/curl/lib/gtls.c.diff?r1=1.26&r2=1.27\n> > Note how gtls_inited is not set back to FALSE in cleanup.\n> \n> Wow. I hope you used \"git bisect\", in order to spare you unnecessary \n> work...\n\nThis was actually easy to spot without. I was pretty sure something\nfishy was going on with the curl_global_cleanup code, so I followed it\nup to Curl_gtls_cleanup and there it was just under my eyes. I only had\nto use the annotate thingy in viewcvs (and leave out version 1.35 of the\nfile, that does whitespace changes only).\n\nMike\n"},{"id":"68018","messageId":"20080208220941.GA22199@glandium.org","threadId":"11871","inReplyTo":"20080208215140.GA21362@glandium.org","subject":"Re: [PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T22:09:41Z","receivedAt":"2008-02-08T22:09:41Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Fri, Feb 08, 2008 at 10:51:40PM +0100, Mike Hommey wrote:\n> On Fri, Feb 08, 2008 at 01:46:14PM -0800, Junio C Hamano wrote:\n> > Mike Hommey writes:\n> > \n> > >> In the end, it was a bit of git's fault, but either curl or gnutls is the\n> > >> actual culprit. I've not looked into either code to find out who's\n> > >> responsible, but a very simplified testcase is as follows:\n> > >> ...\n> > >\n> > > And the winner is... curl !\n> > > The bug was introduced in this commit:\n> > > http://cool.haxx.se/cvs.cgi/curl/lib/gtls.c.diff?r1=1.26&r2=1.27\n> > > Note how gtls_inited is not set back to FALSE in cleanup.\n> > >\n> > > This ended up released in 7.16.3. I'm filing a bug.\n> > \n> > Good detetive work. Thanks.\n> > \n> > I guess we need to ship with a known leak to work this around.\n> > Sigh...\n> \n> We can probably add a test on curl versions to avoid leaking on every\n> install. Something like #if LIBCURL_VERSION_NUM < 0x071003. And then add\n> || LIBCURL_VERSION_NUM > .... whenever this is fixed in curl...\n\n... and 22 minutes after filing the bug, it's fixed in CVS\nhttp://cool.haxx.se/cvs.cgi/curl/lib/gtls.c.diff?r1=1.36&r2=1.37\n\nwhich means it will be fixed in version 7.18.1.\n\nMike\n"},{"id":"68019","messageId":"1202509359-23840-1-git-send-email-mh@glandium.org","threadId":"11871","inReplyTo":"20080208220941.GA22199@glandium.org","subject":"[PATCH v2] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T22:22:39Z","receivedAt":"2008-02-08T22:22:39Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"curl versions 7.16.3 to 7.18.0 included had a regression in which https\nrequests following curl_global_cleanup/init sequence would fail with ASN1\nparser errors with curl-gnutls. Such sequences happen in some cases such\nas git fetch.\n\nWe work around this by removing the http_cleanup call from get_refs_via_curl\nfor broken versions of curl, and allowing http_init to be called several\ntimes without initializing http.c global variables again and leaking old\nvalues, which is a safe thing to have unconditionally.\n\nThe remaining calls to http_cleanup are either last (http-push.c), or almost\nnever called (walker.c; the function it lies in is only called from\ntransport-disconnect, which is called last, and only in builtin-push.c)\nThese leaks shall be addressed in the http code refactoring.\n\nSigned-off-by: Mike Hommey \n---\n http.c | 5 +++++\n transport.c | 2 ++\n 2 files changed, 7 insertions(+), 0 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex d2c11ae..a3aa9e9 100644\n--- a/http.c\n+++ b/http.c\n@@ -215,9 +215,14 @@ static CURL* get_curl_handle(void)\n \n void http_init(void)\n {\n+\tstatic int init = 0;\n \tchar *low_speed_limit;\n \tchar *low_speed_time;\n \n+\tif (init)\n+\t\treturn;\n+\tinit = 1;\n+\n \tcurl_global_init(CURL_GLOBAL_ALL);\n \n \tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\ndiff --git a/transport.c b/transport.c\nindex babaa21..32ab521 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -473,7 +473,9 @@ static struct ref *get_refs_via_curl(struct transport *transport)\n \t\treturn NULL;\n \t}\n \n+#if (LIBCURL_VERSION_NUM < 0x071003) || (LIBCURL_VERSION_NUM > 0x071200)\n \thttp_cleanup();\n+#endif\n \n \tdata = buffer.buf;\n \tstart = NULL;\n-- \n1.5.4.7.gd8534-dirty\n"},{"id":"68025","messageId":"alpine.LSU.1.00.0802082250550.11591@racer.site","threadId":"11871","inReplyTo":"1202509359-23840-1-git-send-email-mh@glandium.org","subject":"Re: [PATCH v2] Work around curl-gnutls not liking to be reinitialized","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-02-08T22:51:49Z","receivedAt":"2008-02-08T22:51:49Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 8 Feb 2008, Mike Hommey wrote:\n\n> diff --git a/http.c b/http.c\n> index d2c11ae..a3aa9e9 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -215,9 +215,14 @@ static CURL* get_curl_handle(void)\n> \n> void http_init(void)\n> {\n> +\tstatic int init = 0;\n> \tchar *low_speed_limit;\n> \tchar *low_speed_time;\n> \n> +\tif (init)\n> +\t\treturn;\n> +\tinit = 1;\n> +\n\nDon't you have to make this conditional on the CURL version as well? I \nmean, that cleanup:\n\n> diff --git a/transport.c b/transport.c\n> index babaa21..32ab521 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -473,7 +473,9 @@ static struct ref *get_refs_via_curl(struct transport *transport)\n> \t\treturn NULL;\n> \t}\n> \n> +#if (LIBCURL_VERSION_NUM < 0x071003) || (LIBCURL_VERSION_NUM > 0x071200)\n> \thttp_cleanup();\n> +#endif\n\nrequires us to init again, no?\n\nCiao,\nDscho\n"},{"id":"68027","messageId":"1202512124-28669-1-git-send-email-mh@glandium.org","threadId":"11871","inReplyTo":"alpine.LSU.1.00.0802082250550.11591@racer.site","subject":"[PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T23:08:44Z","receivedAt":"2008-02-08T23:08:44Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"curl versions 7.16.3 to 7.18.0 included had a regression in which https\nrequests following curl_global_cleanup/init sequence would fail with ASN1\nparser errors with curl-gnutls. Such sequences happen in some cases such\nas git fetch.\n\nWe work around this by removing the http_cleanup call from get_refs_via_curl\nfor the broken versions of curl, and allowing http_init to be called several\ntimes without initializing http.c global variables again and leaking old\nvalues, which is a safe thing to have unconditionally.\n\nThe remaining calls to http_cleanup are either last (http-push.c), or almost\nnever called (walker.c; the function it lies in is only called from\ntransport-disconnect, which is called last, and only in builtin-push.c)\nThese leaks shall be addressed in the http code refactoring.\n\nSigned-off-by: Mike Hommey \n---\n > Don't you have to make this conditional on the CURL version as well? I\n > mean, that cleanup:\n\n > > diff --git a/transport.c b/transport.c\n > > index babaa21..32ab521 100644\n > > --- a/transport.c\n > > +++ b/transport.c\n > > @@ -473,7 +473,9 @@ static struct ref *get_refs_via_curl(struct transport *transport)\n > > return NULL;\n > > }\n > >\n > > +#if (LIBCURL_VERSION_NUM < 0x071003) || (LIBCURL_VERSION_NUM > 0x071200)\n > > http_cleanup();\n > > +#endif\n >\n > requires us to init again, no?\n\n Damn, you're right. But it would actually be better to just have the init\n variable set to 0 again in http_cleanup, and actually, we already have a\n global variable that is set in http_init and reset in http_cleanup that\n could be used for this test...\n\n http.c | 3 +++\n transport.c | 2 ++\n 2 files changed, 5 insertions(+), 0 deletions(-)\n\ndiff --git a/http.c b/http.c\nindex d2c11ae..d69ba90 100644\n--- a/http.c\n+++ b/http.c\n@@ -218,6 +218,9 @@ void http_init(void)\n \tchar *low_speed_limit;\n \tchar *low_speed_time;\n \n+\tif (pragma_header)\n+\t\treturn;\n+\n \tcurl_global_init(CURL_GLOBAL_ALL);\n \n \tpragma_header = curl_slist_append(pragma_header, \"Pragma: no-cache\");\ndiff --git a/transport.c b/transport.c\nindex babaa21..32ab521 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -473,7 +473,9 @@ static struct ref *get_refs_via_curl(struct transport *transport)\n \t\treturn NULL;\n \t}\n \n+#if (LIBCURL_VERSION_NUM < 0x071003) || (LIBCURL_VERSION_NUM > 0x071200)\n \thttp_cleanup();\n+#endif\n \n \tdata = buffer.buf;\n \tstart = NULL;\n-- \n1.5.4.8.g95ac\n"},{"id":"68028","messageId":"20080208231401.GA28920@glandium.org","threadId":"11871","inReplyTo":"1202512124-28669-1-git-send-email-mh@glandium.org","subject":"Re: [PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-08T23:14:01Z","receivedAt":"2008-02-08T23:14:01Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"And I forgot to add a v3 to the title... *sigh*.\n\nSorry\n\nMike\n"},{"id":"68040","messageId":"20080209022759.GD2572@coredump.intra.peff.net","threadId":"11871","inReplyTo":"1202501335-28205-1-git-send-email-mh@glandium.org","subject":"Re: [PATCH] Work around curl-gnutls not liking to be reinitialized","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2008-02-09T02:28:00Z","receivedAt":"2008-02-09T02:28:00Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Feb 08, 2008 at 09:08:55PM +0100, Mike Hommey wrote:\n\n> \t#include \n> \t#include \n> \n> \tint main(void) {\n> \t CURL *easy = curl_easy_init();\n> \t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n> \t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n> \t curl_easy_perform(easy);\n> \t curl_global_cleanup();\n> \t easy = curl_easy_init();\n> \t curl_easy_setopt(easy, CURLOPT_VERBOSE, 1);\n> \t curl_easy_setopt(easy, CURLOPT_URL, \"https://www.verisign.com/\");\n> \t curl_easy_perform(easy);\n> \t}\n\nHrmph. I had tried to produce a similar minimum test case, but for some\nreason I didn't try doing a global_cleanup() between the requests, which\nobviously is the culprit.\n\nThank you for spending the time to track this down. I have confirmed\nthat your fix works on my test case.\n\n-Peff\n"},{"id":"68060","messageId":"1202550096-13233-1-git-send-email-mh@glandium.org","threadId":"11871","inReplyTo":"1202512124-28669-1-git-send-email-mh@glandium.org","subject":"[PATCH v4] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-09T09:41:36Z","receivedAt":"2008-02-09T09:41:36Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"curl versions 7.16.3 to 7.18.0 included had a regression in which https\nrequests following curl_global_cleanup/init sequence would fail with ASN1\nparser errors with curl-gnutls. Such sequences happen in some cases such\nas git fetch.\n\nWe work around this by removing the http_init and http_cleanup calls from\nget_refs_via_curl, replacing them with a transport->data initialization\nwith the http_walker (which does http_init).\n\nWhile the http_walker is not currently used in get_refs_via_curl, http\nand walker code refactor will make it use it.\n\nSigned-off-by: Mike Hommey \n---\n FWIW, the previous patch lacked an initialization for pragma_header. But I\n actually got a better idea ; a more long-term one.\n\n transport.c | 7 +++----\n 1 files changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/transport.c b/transport.c\nindex babaa21..497f853 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -441,11 +441,12 @@ static struct ref *get_refs_via_curl(struct transport *transport)\n \tstruct ref *ref = NULL;\n \tstruct ref *last_ref = NULL;\n \n+\tif (!transport->data)\n+\t\ttransport->data = get_http_walker(transport->url);\n+\n \trefs_url = xmalloc(strlen(transport->url) + 11);\n \tsprintf(refs_url, \"%s/info/refs\", transport->url);\n \n-\thttp_init();\n-\n \tslot = get_active_slot();\n \tslot->results = &results;\n \tcurl_easy_setopt(slot->curl, CURLOPT_FILE, &buffer);\n@@ -473,8 +474,6 @@ static struct ref *get_refs_via_curl(struct transport *transport)\n \t\treturn NULL;\n \t}\n \n-\thttp_cleanup();\n-\n \tdata = buffer.buf;\n \tstart = NULL;\n \tmid = data;\n-- \n1.5.4.35.gb88c\n"},{"id":"68062","messageId":"87y79usbu0.fsf@mid.deneb.enyo.de","threadId":"11871","inReplyTo":"1202509359-23840-1-git-send-email-mh@glandium.org","subject":"Re: [PATCH v2] Work around curl-gnutls not liking to be reinitialized","fromName":"Florian Weimer","fromEmail":"fw@deneb.enyo.de","sentAt":"2008-02-09T09:44:55Z","receivedAt":"2008-02-09T09:44:55Z","isPatch":true,"sender":{"key":"fw@deneb.enyo.de","avatar":null},"body":"* Mike Hommey:\n\n> +#if (LIBCURL_VERSION_NUM < 0x071003) || (LIBCURL_VERSION_NUM > 0x071200)\n> \thttp_cleanup();\n> +#endif\n\nShouldn't you check the version that is used at run time, not the one at\ncompile time?\n"},{"id":"68064","messageId":"20080209104301.GA32309@glandium.org","threadId":"11871","inReplyTo":"87y79usbu0.fsf@mid.deneb.enyo.de","subject":"Re: [PATCH v2] Work around curl-gnutls not liking to be reinitialized","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2008-02-09T10:43:01Z","receivedAt":"2008-02-09T10:43:01Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Sat, Feb 09, 2008 at 10:44:55AM +0100, Florian Weimer wrote:\n> * Mike Hommey:\n> \n> > +#if (LIBCURL_VERSION_NUM < 0x071003) || (LIBCURL_VERSION_NUM > 0x071200)\n> > \thttp_cleanup();\n> > +#endif\n> \n> Shouldn't you check the version that is used at run time, not the one at\n> compile time?\n\nThat is a very good remark, and another good reason to prefer patch v4.\n\nMike\n"},{"id":"68126","messageId":"loom.20080209T214811-305@post.gmane.org","threadId":"11871","inReplyTo":"1202550096-13233-1-git-send-email-mh@glandium.org","subject":"Re: [PATCH v4] Work around curl-gnutls not liking to be reinitialized","fromName":"Daniel Stenberg","fromEmail":"daniel@haxx.se","sentAt":"2008-02-09T21:51:19Z","receivedAt":"2008-02-09T21:51:19Z","isPatch":true,"sender":{"key":"daniel@haxx.se","avatar":"https://gravatar.com/avatar/69fdca87edd17cee21ca2e79fc2ff671d644603c3dc27167430f3cd3dbab7ba8?d=mp&s=160"},"body":"Mike Hommey glandium.org> writes:\n\n> curl versions 7.16.3 to 7.18.0 included had a regression in which https\n> requests following curl_global_cleanup/init sequence would fail with ASN1\n> parser errors with curl-gnutls. Such sequences happen in some cases such\n> as git fetch.\n\nHi git hackers,\n\nI am the main libcurl author and maintainer.\n\nWhile I agree that this is a regression and a bug in libcurl, it puzzles me why\nyou found it in the first place. curl_global_init and curl_global_cleanup should\nonly be needed to call once per program's lifetime. I can't think of any\npractical use to do this re-init at all. You will only waste time on this.\n\nSo, your best fix for this problem is to simply to a curl_global_init() in the\nbeginning and a curl_global_cleanup() in the end.\n\n(I'm not subscribed to this list, just responding via gmane.org)\n"}]}