{"thread":{"id":"41064","subject":"Building Git with HTTPS support: avoiding libcurl?","startedAt":"2015-12-22T15:39:40Z","lastAt":"2015-12-24T22:36:48Z","messageCount":6,"participants":["Paul Smith","Dave Borowitz","Daniel Stenberg","Johannes Schindelin","Thiago Farina"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"274887","messageId":"1450798780.11255.22.camel@mad-scientist.net","threadId":"41064","inReplyTo":null,"subject":"Building Git with HTTPS support: avoiding libcurl?","fromName":"Paul Smith","fromEmail":"paul@mad-scientist.net","sentAt":"2015-12-22T15:39:40Z","receivedAt":"2015-12-22T15:39:40Z","isPatch":false,"sender":{"key":"paul@mad-scientist.net","avatar":"https://avatars.githubusercontent.com/u/109636?v=4"},"body":"I'm trying to build Git (2.6.4) on GNU/Linux, but without any\nrequirements (other than basic libc etc.) on the local system. This\nworks fine except for one thing: git-remote-https.\n\nIn order to build this I need to have libcurl, but libcurl is a MONSTER\nlibrary with an enormous number of prerequisites (see below).\n\nJust wondering if anyone has considered an alternative to libcurl; maybe\nI'm wrong but it seems to me that HTTPS support for Git would require\nonly a tiny fraction of the libcurl features and maybe there's an\nalternative available which would be more targeted?\n\nI realize this is not a short-term thing in that there won't be an API\ncompatible library that can just be dropped in. This is more a forward\n-looking question. For now I'm looking to see if I can rebuild libcurl\nmyself without most of these dependencies such as Kerberos, LDAP, etc.\n\n\n$ ldd /usr/lib/x86_64-linux-gnu/libcurl.so.4\n linux-vdso.so.1 => (0x00007fff37d81000)\n libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007f682b921000)\n librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f682b704000)\n libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f682b49a000)\n libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f682b058000)\n libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f682ae0e000)\n liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f682abfe000)\n libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f682a9ac000)\n libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f682a792000)\n libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f682a573000)\n libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f682a1a9000)\n libgnutls-deb0.so.28 => /usr/lib/x86_64-linux-gnu/libgnutls-deb0.so.28 (0x00007f6829e8d000)\n libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f6829c59000)\n libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f6829a23000)\n libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f68297a3000)\n libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f682959e000)\n libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f68292cc000)\n libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f682909d000)\n libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f6828e98000)\n libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f6828c8d000)\n libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f6828a71000)\n libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f6828855000)\n libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f6828615000)\n /lib64/ld-linux-x86-64.so.2 (0x0000559b03259000)\n libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f68283b0000)\n libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f682819c000)\n libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f6827f98000)\n libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f6827d8e000)\n libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f6827b04000)\n libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f6827861000)\n libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f682762d000)\n libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007f6827418000)\n libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f6827210000)\n libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007f6826fe6000)\n libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f6826dd7000)\n libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f6826b8c000)\n libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f68268be000)\n libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f6826686000)\n"},{"id":"274896","messageId":"CAD0k6qT+s4e_7y1DxVTN63V0tO_xFv-9i-Fmq5O0SrpQAyAzVA@mail.gmail.com","threadId":"41064","inReplyTo":"1450798780.11255.22.camel@mad-scientist.net","subject":"Re: Building Git with HTTPS support: avoiding libcurl?","fromName":"Dave Borowitz","fromEmail":"dborowitz@google.com","sentAt":"2015-12-22T17:08:00Z","receivedAt":"2015-12-22T17:08:00Z","isPatch":false,"sender":{"key":"dborowitz@google.com","avatar":"https://avatars.githubusercontent.com/u/194927?v=4"},"body":"On Tue, Dec 22, 2015 at 7:39 AM, Paul Smith wrote:\n> I'm trying to build Git (2.6.4) on GNU/Linux, but without any\n> requirements (other than basic libc etc.) on the local system. This\n> works fine except for one thing: git-remote-https.\n>\n> In order to build this I need to have libcurl, but libcurl is a MONSTER\n> library with an enormous number of prerequisites (see below).\n\nWell, IIUC one of the reasons for Git's fork-everything strategy is to\navoid having to dynamically link the core git binary against large\nlibraries like libcurl, so the dependency size has been taken into\naccount at least in that sense.\n\n> Just wondering if anyone has considered an alternative to libcurl; maybe\n> I'm wrong but it seems to me that HTTPS support for Git would require\n> only a tiny fraction of the libcurl features and maybe there's an\n> alternative available which would be more targeted?\n>\n> I realize this is not a short-term thing in that there won't be an API\n> compatible library that can just be dropped in. This is more a forward\n> -looking question. For now I'm looking to see if I can rebuild libcurl\n> myself without most of these dependencies such as Kerberos, LDAP, etc.\n>\n>\n> $ ldd /usr/lib/x86_64-linux-gnu/libcurl.so.4\n> linux-vdso.so.1 => (0x00007fff37d81000)\n> libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007f682b921000)\n> librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f682b704000)\n> libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f682b49a000)\n> libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f682b058000)\n> libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f682ae0e000)\n> liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f682abfe000)\n> libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f682a9ac000)\n> libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f682a792000)\n> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f682a573000)\n> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f682a1a9000)\n> libgnutls-deb0.so.28 => /usr/lib/x86_64-linux-gnu/libgnutls-deb0.so.28 (0x00007f6829e8d000)\n> libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f6829c59000)\n> libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f6829a23000)\n> libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f68297a3000)\n> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f682959e000)\n> libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f68292cc000)\n> libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f682909d000)\n> libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f6828e98000)\n> libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f6828c8d000)\n> libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f6828a71000)\n> libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f6828855000)\n> libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f6828615000)\n> /lib64/ld-linux-x86-64.so.2 (0x0000559b03259000)\n> libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f68283b0000)\n> libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f682819c000)\n> libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f6827f98000)\n> libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f6827d8e000)\n> libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f6827b04000)\n> libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f6827861000)\n> libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f682762d000)\n> libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007f6827418000)\n> libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f6827210000)\n> libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007f6826fe6000)\n> libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f6826dd7000)\n> libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f6826b8c000)\n> libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f68268be000)\n> libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f6826686000)\n\nMaybe half of those dependencies are crypto libraries, so even if you\nmanaged to eliminate libcurl, you'd have a hard time supporting HTTPS\nwithout them. Or maybe use something like boringssl instead?\n"},{"id":"274898","messageId":"1450805416.11255.58.camel@mad-scientist.net","threadId":"41064","inReplyTo":"CAD0k6qT+s4e_7y1DxVTN63V0tO_xFv-9i-Fmq5O0SrpQAyAzVA@mail.gmail.com","subject":"Re: Building Git with HTTPS support: avoiding libcurl?","fromName":"Paul Smith","fromEmail":"paul@mad-scientist.net","sentAt":"2015-12-22T17:30:16Z","receivedAt":"2015-12-22T17:30:16Z","isPatch":false,"sender":{"key":"paul@mad-scientist.net","avatar":"https://avatars.githubusercontent.com/u/109636?v=4"},"body":"On Tue, 2015-12-22 at 09:08 -0800, Dave Borowitz wrote:\n> Well, IIUC one of the reasons for Git's fork-everything strategy is to\n> avoid having to dynamically link the core git binary against large\n> libraries like libcurl, so the dependency size has been taken into\n> account at least in that sense.\n\nSure, and it's great that I still get most Git even if I don't have\nlibcurl.\n\nBut without support for cloning https remotes there's a significant hole\nin Git functionality...\n\nI grok that Git doesn't want to re-invent the wheel and that libcurl is\nconvenient. I just wonder if anyone knows of another wheel, that\ndoesn't come attached to an entire tractor-trailer, that could be used\ninstead :).\n"},{"id":"274939","messageId":"alpine.DEB.2.11.1512231059380.23702@tvnag.unkk.fr","threadId":"41064","inReplyTo":"1450805416.11255.58.camel@mad-scientist.net","subject":"Re: Building Git with HTTPS support: avoiding libcurl?","fromName":"Daniel Stenberg","fromEmail":"daniel@haxx.se","sentAt":"2015-12-23T10:17:35Z","receivedAt":"2015-12-23T10:17:35Z","isPatch":false,"sender":{"key":"daniel@haxx.se","avatar":"https://gravatar.com/avatar/69fdca87edd17cee21ca2e79fc2ff671d644603c3dc27167430f3cd3dbab7ba8?d=mp&s=160"},"body":"On Tue, 22 Dec 2015, Paul Smith wrote:\n\n> I grok that Git doesn't want to re-invent the wheel and that libcurl is \n> convenient. I just wonder if anyone knows of another wheel, that doesn't \n> come attached to an entire tractor-trailer, that could be used instead :).\n\nBut if you would consider another lib, then you could just rebuild your own \nlibcurl instead from source, entirely without any dependencies on other libs! \nThat would be similar to finding another lib with less dependencies. (As \nalready mentioned, you'd still need crypto and TLS support no doubt.)\n\nThat huge dependency collection is there much because your distro decided that \nhaving a libcurl with all that support is preferable. libcurl itself offers \nlots of customizability at build-time so you can strip out most of that if you \nwanted to.\n\nBut why do the distros build and provide a libcurl that can do all this?\n\nI think you can look at this from a slightly higher altitude. By re-using a \nvery widely used, well developed and properly documented library (yeah, I \nclaim it is but you don't need to take my word for it) that is available \neverywhere - git benefits. By having many projects use the same lib, even if \nno two projects use the exact same feature set, we get more reliable software \nin the entire ecosystem - with less work.\n\nI would guess that switching out libcurl in git would be a not insignificant \namount of work, as no libcurl alternative I'm aware of is even close to this \nAPI.\n\n-- \n\n / daniel.haxx.se\n"},{"id":"274958","messageId":"alpine.DEB.2.20.1512232033170.14434@virtualbox","threadId":"41064","inReplyTo":"alpine.DEB.2.11.1512231059380.23702@tvnag.unkk.fr","subject":"Re: Building Git with HTTPS support: avoiding libcurl?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2015-12-23T19:35:52Z","receivedAt":"2015-12-23T19:35:52Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Daniel,\n\nOn Wed, 23 Dec 2015, Daniel Stenberg wrote:\n\n> By re-using a very widely used, well developed and properly documented\n> library [libcurl] (yeah, I claim it is but you don't need to take my\n> word for it) that is available everywhere - git benefits.\n\nFor what it's worth, I fully agree. My perspective: By using a pretty much\nfully-configured cURL, Git for Windows has supported the largest number of\nuse cases with minimal requirements from myself. It has been a boon. If\nevery library was as easy to use and as high quality, anybody could\nmaintain Git for Windows ;-)\n\nCiao,\na very grateful Dscho\n"},{"id":"274994","messageId":"CACnwZYdLS7F-crNu0RjdYw__82nbaTujnjDc+q2jAXZ2Qoja+Q@mail.gmail.com","threadId":"41064","inReplyTo":"1450798780.11255.22.camel@mad-scientist.net","subject":"Re: Building Git with HTTPS support: avoiding libcurl?","fromName":"Thiago Farina","fromEmail":"tfransosi@gmail.com","sentAt":"2015-12-24T22:36:48Z","receivedAt":"2015-12-24T22:36:48Z","isPatch":false,"sender":{"key":"tfransosi@gmail.com","avatar":"https://avatars.githubusercontent.com/u/970071?v=4"},"body":"On Tue, Dec 22, 2015 at 1:39 PM, Paul Smith wrote:\n> I'm trying to build Git (2.6.4) on GNU/Linux, but without any\n> requirements (other than basic libc etc.) on the local system. This\n> works fine except for one thing: git-remote-https.\n>\n> In order to build this I need to have libcurl, but libcurl is a MONSTER\n> library with an enormous number of prerequisites (see below).\n>\nI think Git would have to be changed to use raw sockets and implement\neverything it needs on top of that, like libgit2 already does. Certainly this\nwon't be a trivial task.\n\n-- \nThiago Farina\n"}]}