{"thread":{"id":"42691","subject":"Use docker for _some_ testing?","startedAt":"2016-06-22T19:02:03Z","lastAt":"2016-06-23T04:14:51Z","messageCount":4,"participants":["Duy Nguyen","Junio C Hamano","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"289887","messageId":"20160622190154.GA17442@duynguyen","threadId":"42691","inReplyTo":null,"subject":"Use docker for _some_ testing?","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-06-22T19:01:55Z","receivedAt":"2016-06-22T19:02:03Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"The story started with my problem on Debian, which I didn't have and\ndidn't want to install a Debian VM just for that problem. So I made a\ndocker image with the following script.\n\nWhich makes me think, could we use something like this to make sure\npeople (on Linux) can test more obscure cases? Sometimes there are\nfeatues that require some dependencies that are not always present on\nthe developer's machine (http server is a big one, locales come close\nsecond, then there will be lmdb and watchman in future...). With this,\nsaid developer can do a final test run in docker covering as much as\npossible.\n\nOf course it can't cover everything. Different compiler versions are\nout. OS-specific changes are out (but wine would be still good to test\nsome aspect of Windows port, or at least make sure it builds)\n\nComments?\n\n-- 8< --\ndiff --git a/contrib/docker/locale.gen b/contrib/docker/locale.gen\nnew file mode 100644\nindex 0000000..ef08e00\n--- /dev/null\n+++ b/contrib/docker/locale.gen\n@@ -0,0 +1,2 @@\n+is_IS.UTF-8 UTF-8\n+is_IS ISO-8859-1\n\\ No newline at end of file\ndiff --git a/contrib/docker/run.sh b/contrib/docker/run.sh\nnew file mode 100755\nindex 0000000..83e5679\n--- /dev/null\n+++ b/contrib/docker/run.sh\n@@ -0,0 +1,30 @@\n+#!/bin/sh\n+\n+die() {\n+\techo \"$@\" >&2\n+\texit 1\n+}\n+\n+build_debian() {\n+\tcat >Dockerfile <<-EOF\n+\tFROM debian:latest\n+\tRUN apt-get update && \\\n+\t\tapt-get install -y libcurl4-gnutls-dev libexpat1-dev \\\n+\t\tgettext libz-dev libssl-dev build-essential\n+\tRUN apt-get install -y locales\n+\tCOPY locale.gen /etc/locale.gen\n+\tRUN locale-gen\n+\tRUN groupadd -r $(id -gn) -g $(id -g) && \\\n+\t\tuseradd -u $(id -u) -r -d \"$HOME\" -g $(id -g) -s /sbin/nologin $(id -un)\n+\tUSER $(id -un)\n+\tEOF\n+\tdocker build -t $IMAGE .  || die \"failed to build docker image\"\n+}\n+\n+DISTRO=debian\n+IMAGE=git-$DISTRO-$(id -un)\n+ROOT=\"$(realpath $(git rev-parse --show-cdup))\"\n+\n+test \"$(docker images --format='{{.Repository}}' $IMAGE)\" = $IMAGE || \\\n+\tbuild_$DISTRO\n+docker run -it --rm -v \"$ROOT\":\"$ROOT\" -w \"$(pwd)\" $IMAGE bash\n-- 8< --\n--\nDuy\n"},{"id":"289891","messageId":"xmqqziqdkoi7.fsf@gitster.mtv.corp.google.com","threadId":"42691","inReplyTo":"20160622190154.GA17442@duynguyen","subject":"Re: Use docker for _some_ testing?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-06-22T19:33:04Z","receivedAt":"2016-06-22T19:33:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen <pclouds@gmail.com> writes:\n\n> The story started with my problem on Debian, which I didn't have and\n> didn't want to install a Debian VM just for that problem. So I made a\n> docker image with the following script.\n>\n> Which makes me think, could we use something like this to make sure\n> people (on Linux) can test more obscure cases? Sometimes there are\n> featues that require some dependencies that are not always present on\n> the developer's machine (http server is a big one, locales come close\n> second, then there will be lmdb and watchman in future...). With this,\n> said developer can do a final test run in docker covering as much as\n> possible.\n>\n> Of course it can't cover everything. Different compiler versions are\n> out. OS-specific changes are out (but wine would be still good to test\n> some aspect of Windows port, or at least make sure it builds)\n>\n> Comments?\n\nNice.\n\n> -- 8< --\n> diff --git a/contrib/docker/locale.gen b/contrib/docker/locale.gen\n> new file mode 100644\n> index 0000000..ef08e00\n> --- /dev/null\n> +++ b/contrib/docker/locale.gen\n> @@ -0,0 +1,2 @@\n> +is_IS.UTF-8 UTF-8\n> +is_IS ISO-8859-1\n> \\ No newline at end of file\n> diff --git a/contrib/docker/run.sh b/contrib/docker/run.sh\n> new file mode 100755\n> index 0000000..83e5679\n> --- /dev/null\n> +++ b/contrib/docker/run.sh\n> @@ -0,0 +1,30 @@\n> +#!/bin/sh\n> +\n> +die() {\n> +\techo \"$@\" >&2\n> +\texit 1\n> +}\n> +\n> +build_debian() {\n> +\tcat >Dockerfile <<-EOF\n> +\tFROM debian:latest\n> +\tRUN apt-get update && \\\n> +\t\tapt-get install -y libcurl4-gnutls-dev libexpat1-dev \\\n> +\t\tgettext libz-dev libssl-dev build-essential\n> +\tRUN apt-get install -y locales\n> +\tCOPY locale.gen /etc/locale.gen\n> +\tRUN locale-gen\n> +\tRUN groupadd -r $(id -gn) -g $(id -g) && \\\n> +\t\tuseradd -u $(id -u) -r -d \"$HOME\" -g $(id -g) -s /sbin/nologin $(id -un)\n> +\tUSER $(id -un)\n> +\tEOF\n> +\tdocker build -t $IMAGE .  || die \"failed to build docker image\"\n> +}\n> +\n> +DISTRO=debian\n> +IMAGE=git-$DISTRO-$(id -un)\n> +ROOT=\"$(realpath $(git rev-parse --show-cdup))\"\n> +\n> +test \"$(docker images --format='{{.Repository}}' $IMAGE)\" = $IMAGE || \\\n> +\tbuild_$DISTRO\n> +docker run -it --rm -v \"$ROOT\":\"$ROOT\" -w \"$(pwd)\" $IMAGE bash\n> -- 8< --\n> --\n> Duy\n"},{"id":"289895","messageId":"20160622195932.GC1697@sigill.intra.peff.net","threadId":"42691","inReplyTo":"20160622190154.GA17442@duynguyen","subject":"Re: Use docker for _some_ testing?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-06-22T19:59:32Z","receivedAt":"2016-06-22T19:59:37Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jun 22, 2016 at 09:01:55PM +0200, Duy Nguyen wrote:\n\n> Which makes me think, could we use something like this to make sure\n> people (on Linux) can test more obscure cases? Sometimes there are\n> featues that require some dependencies that are not always present on\n> the developer's machine (http server is a big one, locales come close\n> second, then there will be lmdb and watchman in future...). With this,\n> said developer can do a final test run in docker covering as much as\n> possible.\n\nNeat idea. This should also cover some other distro-specific issues like\n\"what's your /bin/sh look like\", etc. It's a lighter-weight alternative\nto testing alternate distros in a VM.\n\nBut I think most of the interesting cases are more exotic than\ndistro-to-distro. Things like HFS+ normalization, or weird shell\ntoolchains on proprietary Unixes (but maybe there are docker images for\nthose systems?).\n\nSo I dunno if it would prove that useful for day to day use or not.\n\n> +ROOT=\"$(realpath $(git rev-parse --show-cdup))\"\n\nI tried running this as contrib/docker/run.sh, which complained here,\nbecause --show-cdup is empty.\n\n> +test \"$(docker images --format='{{.Repository}}' $IMAGE)\" = $IMAGE || \\\n> +\tbuild_$DISTRO\n\nMy docker complained that it doesn't know --format. I guess I might just\nhave an old one (it's whatever is in Debian unstable, which is 1.8.3).\n\nNot things you need to fix, but just comments for anybody else fiddling\nwith it.\n\n-Peff\n"},{"id":"289939","messageId":"CACsJy8Dig1ENn7A5RsMd7YqqYEKOX5xXVh8ahFQy-KjW+m_nWQ@mail.gmail.com","threadId":"42691","inReplyTo":"20160622195932.GC1697@sigill.intra.peff.net","subject":"Re: Use docker for _some_ testing?","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-06-23T04:14:18Z","receivedAt":"2016-06-23T04:14:51Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Jun 22, 2016 at 9:59 PM, Jeff King <peff@peff.net> wrote:\n> On Wed, Jun 22, 2016 at 09:01:55PM +0200, Duy Nguyen wrote:\n>\n>> Which makes me think, could we use something like this to make sure\n>> people (on Linux) can test more obscure cases? Sometimes there are\n>> featues that require some dependencies that are not always present on\n>> the developer's machine (http server is a big one, locales come close\n>> second, then there will be lmdb and watchman in future...). With this,\n>> said developer can do a final test run in docker covering as much as\n>> possible.\n>\n> Neat idea. This should also cover some other distro-specific issues like\n> \"what's your /bin/sh look like\", etc. It's a lighter-weight alternative\n> to testing alternate distros in a VM.\n\nIt's lighter-weight and also helps skip preparation. Everything is\nscripted, if you want distro X, it's just one command away. The VM way\nwould require you to go through installation process with lots of\nclicking and typing (unless you go full cloud solution like OpenStack,\nnot really suited for single dev use).\n\nIt's not just distro-specific. Not every developer has enough stuff\ninstalled to run all tests. My machine is svn-free for example so I\nnever run those tests. Same story for pcre (have it but not enable it\nin git...). We can try to run both gettext and no-gettext\nconfiguration... Devs still have to install docker this way, but at\nleast i could keep my laptop \"clean\" from unwanted stuff and still be\nable to run lots of tests.\n\n> But I think most of the interesting cases are more exotic than\n> distro-to-distro. Things like HFS+ normalization, or weird shell\n> toolchains on proprietary Unixes (but maybe there are docker images for\n> those systems?).\n\nI don't follow docker closely, but if it's still a container\ntechnology (i.e. sharing host kernel) then different OSes are out of\nquestion.\n\n> So I dunno if it would prove that useful for day to day use or not.\n-- \nDuy\n"}]}