{"thread":{"id":"56682","subject":"[PATCH] archive: add test testing MIME for created archive through compression filter","startedAt":"2021-10-11T11:28:31Z","lastAt":"2021-10-12T19:15:43Z","messageCount":6,"participants":["Bagas Sanjaya","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"438452","messageId":"20211011112712.475306-1-bagasdotme@gmail.com","threadId":"56682","inReplyTo":null,"subject":"[PATCH] archive: add test testing MIME for created archive through compression filter","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-10-11T11:27:13Z","receivedAt":"2021-10-11T11:28:31Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"The common use of `tar..command` config is to specify\ncompression program filter for creating compressed tar archive.\n\nAdd a test that tests MIME type of archives created through the filter.\nThe generated archives must not be `application/x-tar` (POSIX tar\narchive) type, so that these can be decompressed with the corresponding\ndecompression program (such as gunzip, bunzip2, and unxz).\n\nSigned-off-by: Bagas Sanjaya \n---\n t/t5000-tar-tree.sh | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh\nindex 2c88d1c159..04cdad5bff 100755\n--- a/t/t5000-tar-tree.sh\n+++ b/t/t5000-tar-tree.sh\n@@ -38,6 +38,8 @@ test_lazy_prereq TAR_NEEDS_PAX_FALLBACK '\n '\n \n test_lazy_prereq GZIP 'gzip --version'\n+test_lazy_prereq BZIP2 'bzip2 --version'\n+test_lazy_prereq XZ 'xz --version'\n \n get_pax_header() {\n \tfile=$1\n@@ -374,6 +376,21 @@ test_expect_success GZIP 'remote tar.gz can be disabled' '\n \t\t>remote.tar.gz\n '\n \n+test_expect_success GZIP,BZIP2,XZ 'git archive with gzip, bzip2, and xz filters creates compressed tar archive with proper MIME type' '\n+\tgit config tar.tar.bz2.command \"bzip2 -c\" &&\n+\tgit config tar.tar.xz.command \"xz -c\" &&\n+\tgit archive --output HEAD.tar.gz --prefix=src/ HEAD &&\n+\tgit archive --output HEAD.tar.bz2 --prefix=src/ HEAD &&\n+\tgit archive --output HEAD.tar.xz --prefix=src/ HEAD &&\n+\tcat >expect </dev/null >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'archive and :(glob)' '\n \tgit archive -v HEAD -- \":(glob)**/sh\" >/dev/null 2>actual &&\n \tcat >expect < writes:\n\n> +\tcat >expect < +HEAD.tar.bz2: application/x-bzip2; charset=binary\n> +HEAD.tar.gz: application/gzip; charset=binary\n> +HEAD.tar.xz: application/x-xz; charset=binary\n> +EOF\n\nUse <<-EOF and you can align here-document to the script.\n\n> +\tfile -i HEAD.tar.bz2 HEAD.tar.gz HEAD.tar.xz 2>/dev/null >actual &&\n\nThis looks super un-portable.\n\n https://pubs.opengroup.org/onlinepubs/9699919799/utilities/file.html\n\n\n"},{"id":"438470","messageId":"xmqqlf2zd0d4.fsf@gitster.g","threadId":"56682","inReplyTo":"20211011112712.475306-1-bagasdotme@gmail.com","subject":"Re: [PATCH] archive: add test testing MIME for created archive through compression filter","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-10-11T15:53:11Z","receivedAt":"2021-10-11T15:53:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Bagas Sanjaya writes:\n\n> +test_expect_success GZIP,BZIP2,XZ 'git archive with gzip, bzip2, and xz filters creates compressed tar archive with proper MIME type' '\n> +\tgit config tar.tar.bz2.command \"bzip2 -c\" &&\n> +\tgit config tar.tar.xz.command \"xz -c\" &&\n> +\tgit archive --output HEAD.tar.gz --prefix=src/ HEAD &&\n\nI think a lot more portable and robust way to test the feature is to\nconfigure git config tar.tar.test.command (or come up with a name for\na suffix to be used in the test), point it at a script created in this\ntest script and run git archive with output filename that would trigger\nthe command. Then, arrange the test to notice if the \"test\" script was\ncalled with expected command line arguments and standard input.\n\nThat way, you do not need to rely on prereqs and you do not have to\nresort to un-portable use of the \"file\" command. After all, you are\n*not* testing if \"bzip2 -c\" the user happens to have on their $PATH\nproduces output their \"find\" recognises as bzip2 compressed.\n\n"},{"id":"438473","messageId":"YWRhR9IXZfU93Uok@coredump.intra.peff.net","threadId":"56682","inReplyTo":"xmqqlf2zd0d4.fsf@gitster.g","subject":"Re: [PATCH] archive: add test testing MIME for created archive through compression filter","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-10-11T16:07:35Z","receivedAt":"2021-10-11T16:07:38Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Oct 11, 2021 at 08:53:11AM -0700, Junio C Hamano wrote:\n\n> Bagas Sanjaya writes:\n> \n> > +test_expect_success GZIP,BZIP2,XZ 'git archive with gzip, bzip2, and xz filters creates compressed tar archive with proper MIME type' '\n> > +\tgit config tar.tar.bz2.command \"bzip2 -c\" &&\n> > +\tgit config tar.tar.xz.command \"xz -c\" &&\n> > +\tgit archive --output HEAD.tar.gz --prefix=src/ HEAD &&\n> \n> I think a lot more portable and robust way to test the feature is to\n> configure git config tar.tar.test.command (or come up with a name for\n> a suffix to be used in the test), point it at a script created in this\n> test script and run git archive with output filename that would trigger\n> the command. Then, arrange the test to notice if the \"test\" script was\n> called with expected command line arguments and standard input.\n> \n> That way, you do not need to rely on prereqs and you do not have to\n> resort to un-portable use of the \"file\" command. After all, you are\n> *not* testing if \"bzip2 -c\" the user happens to have on their $PATH\n> produces output their \"find\" recognises as bzip2 compressed.\n\nThat would work. Or perhaps using something portable like \"tr\" to\ndo a nonsense conversion, and verifying that we can get convert back to\nthe original. Like say, the tests added to t5000 by 767cf4579f (archive:\nimplement configurable tar filters, 2011-06-21).\n\nI don't think the newly proposed test is adding anything beyond that\n(except checking the system \"bzip2\" command, but as you say, that is not\nuseful to us, nor portable).\n\n-Peff\n"},{"id":"438516","messageId":"fab92b70-8004-5765-1b8a-2cf1d12abb63@gmail.com","threadId":"56682","inReplyTo":"xmqqlf2zd0d4.fsf@gitster.g","subject":"Re: [PATCH] archive: add test testing MIME for created archive through compression filter","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-10-12T03:14:46Z","receivedAt":"2021-10-12T03:14:54Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 11/10/21 22.53, Junio C Hamano wrote:\n> I think a lot more portable and robust way to test the feature is to\n> configure git config tar.tar.test.command (or come up with a name for\n> a suffix to be used in the test), point it at a script created in this\n> test script and run git archive with output filename that would trigger\n> the command. Then, arrange the test to notice if the \"test\" script was\n> called with expected command line arguments and standard input.\n> \n> That way, you do not need to rely on prereqs and you do not have to\n> resort to un-portable use of the \"file\" command. After all, you are\n> *not* testing if \"bzip2 -c\" the user happens to have on their $PATH\n> produces output their \"find\" recognises as bzip2 compressed.\n> \n\nThe intent of this test is to ensure `git archive -o \nsomething.tar.` produces proper compressed tar archive that can \nbe decompressed with the corresponding decompression tool (such as \ngunzip for gz files), and not just with `tar xvf`.\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"438584","messageId":"xmqq35p63vhg.fsf@gitster.g","threadId":"56682","inReplyTo":"fab92b70-8004-5765-1b8a-2cf1d12abb63@gmail.com","subject":"Re: [PATCH] archive: add test testing MIME for created archive through compression filter","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-10-12T19:15:39Z","receivedAt":"2021-10-12T19:15:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Bagas Sanjaya writes:\n\n> On 11/10/21 22.53, Junio C Hamano wrote:\n>> I think a lot more portable and robust way to test the feature is to\n>> configure git config tar.tar.test.command (or come up with a name for\n>> a suffix to be used in the test), point it at a script created in this\n>> test script and run git archive with output filename that would trigger\n>> the command. Then, arrange the test to notice if the \"test\" script was\n>> called with expected command line arguments and standard input.\n>> That way, you do not need to rely on prereqs and you do not have to\n>> resort to un-portable use of the \"file\" command. After all, you are\n>> *not* testing if \"bzip2 -c\" the user happens to have on their $PATH\n>> produces output their \"find\" recognises as bzip2 compressed.\n>> \n>\n> The intent of this test is to ensure `git archive -o\n> something.tar.` produces proper compressed tar archive that\n> can be decompressed with the corresponding decompression tool (such as \n> gunzip for gz files), and not just with `tar xvf`.\n\nYes, that is why relying on \"file -i\" is not the right thing to do.\n\nWe'd want to use the matching decompressor, or better yet, trust\nthat the compressor-decompressor pair the system offers do work\ncorrectly (after all, it's not our test suite's job to debug the\nsystem software the tester uses) and validate that\n\n (1) the specified compressor is spawned with the right parameters\n taken from the configuration file, and that\n\n (2) the compressor is fed the correct input stream.\n\nUsing a script prepared by the test itself as the \"compressor\" would\nbe one good way to validate (1); it can record how it was invoked,\nso that the test can check its correctness. Using a reversible\ntransformation in the script as a fake compressor and comparing the\nreversed output with the output without any compression filter, may\nbe one way to validate (2), like Peff mentioned.\n\nBut asking \"file -i\" validates neither.\n\nThe code that reads the configuration and spawns the compressor with\nspecified parameters may be broken and may not pass the right\nparameters, or the archive stream may not be passed to the\ncompressor intact. But the compressor may still produce a header\nblock that is looking correct enough for the \"file\" command to say\n\"ah, that uses X compressor\", and the tests in the patch under\ndiscussion does not even see if the payload was passed correctly\nwithout corruption to the compressor at all.\n"}]}