{"thread":{"id":"45981","subject":"reversion in GIT_COMMON_DIR refs path","startedAt":"2017-05-16T17:17:21Z","lastAt":"2017-05-22T11:12:02Z","messageCount":14,"participants":["Joey Hess","Ævar Arnfjörð Bjarmason","Junio C Hamano","Stefan Beller","Brandon Williams","Duy Nguyen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"319935","messageId":"20170516171028.5eagqr2sw5a2i77d@kitenet.net","threadId":"45981","inReplyTo":null,"subject":"reversion in GIT_COMMON_DIR refs path","fromName":"Joey Hess","fromEmail":"id@joeyh.name","sentAt":"2017-05-16T17:10:28Z","receivedAt":"2017-05-16T17:17:21Z","isPatch":false,"sender":{"key":"id@joeyh.name","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"Bisecting this test suite failure\nhttps://git-annex.branchable.com/git-annex_in_nixpkgs_fails_with_git-2.13.0/\nI landed on commit f57f37e2e1bf11ab4cdfd221ad47e961ba9353a0 to git.\n\nIt seems that changed resolving refs paths when GIT_DIR and GIT_COMMON_DIR\nare both set. While before refs were looked for in GIT_COMMON_DIR,\nnow they're not.\n\nTest case:\n\n#!/bin/sh\nset -e\nset -x\nrm -rf testdir\ngit init testdir\ncd testdir\necho 1 > foo\ngit add foo\ngit commit -m add\nmkdir dummy\nmkdir dummy/overlay\ncp .git/index .git/HEAD dummy/overlay\n#cp .git/refs .git/packed-refs dummy/overlay -a\ncd dummy\nexport GIT_COMMON_DIR=`pwd`/../.git\nexport GIT_DIR=`pwd`/overlay\ngit rev-parse --git-path refs/heads/master\ngit show refs/heads/master\n\nThis script succeeds with git 2.11.0, but with 2.13.0, it fails:\n\nfatal: ambiguous argument 'refs/heads/master': unknown revision or path not in the working tree.\n\nIt seems to be failing to look up refs in GIT_COMMON_DIR.\nNote that uncommenting the commented out line in the script, to copy the refs\ninto GIT_DIR, makes it succeed.\n\ngit rev-parse --git-path refs/heads/master shows the GIT_COMMON_DIR/refs path\nstill (as gitrepository-layout documents). So this reversion made\ndifferent parts of git disagreeing about the refs path.\n"},{"id":"319946","messageId":"CACBZZX5AgdMTceBAVftNp0goHpf2-hqx8GzvJshx2n1FtCGsBw@mail.gmail.com","threadId":"45981","inReplyTo":"20170516171028.5eagqr2sw5a2i77d@kitenet.net","subject":"Re: reversion in GIT_COMMON_DIR refs path","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-16T17:50:27Z","receivedAt":"2017-05-16T17:50:53Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, May 16, 2017 at 7:10 PM, Joey Hess <id@joeyh.name> wrote:\n> Bisecting this test suite failure\n> https://git-annex.branchable.com/git-annex_in_nixpkgs_fails_with_git-2.13.0/\n> I landed on commit f57f37e2e1bf11ab4cdfd221ad47e961ba9353a0 to git.\n\nThat links's broken for me. Looking at your wiki it looks like you\nmean: https://git-annex.branchable.com/bugs/git-annex_in_nixpkgs_fails_with_git-2.13.0/\n\nI have no idea what this bug is about, but side-question: It looks\nlike this is git-annex's own test suite that's failing with 2.13.0, is\nthat right?\n\nIt would be very nice to have a test in git itself to test with\ngit-annex. I.e. some optional test that just pulls down the latest\ngit-annex release & runs its tests against the git we're building.\n\nThanks for annex b.t.w., I use it a lot.\n"},{"id":"319947","messageId":"20170516175906.hdwn4x5md7dj7fo3@kitenet.net","threadId":"45981","inReplyTo":"CACBZZX5AgdMTceBAVftNp0goHpf2-hqx8GzvJshx2n1FtCGsBw@mail.gmail.com","subject":"Re: reversion in GIT_COMMON_DIR refs path","fromName":"Joey Hess","fromEmail":"id@joeyh.name","sentAt":"2017-05-16T17:59:06Z","receivedAt":"2017-05-16T17:59:19Z","isPatch":false,"sender":{"key":"id@joeyh.name","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"Ævar Arnfjörð Bjarmason wrote:\n> On Tue, May 16, 2017 at 7:10 PM, Joey Hess <id@joeyh.name> wrote:\n> > Bisecting this test suite failure\n> > https://git-annex.branchable.com/git-annex_in_nixpkgs_fails_with_git-2.13.0/\n> > I landed on commit f57f37e2e1bf11ab4cdfd221ad47e961ba9353a0 to git.\n> \n> That links's broken for me. Looking at your wiki it looks like you\n> mean: https://git-annex.branchable.com/bugs/git-annex_in_nixpkgs_fails_with_git-2.13.0/\n\nThanks for correcting that\n\n> I have no idea what this bug is about, but side-question: It looks\n> like this is git-annex's own test suite that's failing with 2.13.0, is\n> that right?\n\nYes indeed.\n\n> It would be very nice to have a test in git itself to test with\n> git-annex. I.e. some optional test that just pulls down the latest\n> git-annex release & runs its tests against the git we're building.\n> \n> Thanks for annex b.t.w., I use it a lot.\n\nIf the git devs are ok with this, I certianly would be happy if such\ntests were run, at least occasionally, on the git side!\n\n-- \nsee shy jo\n"},{"id":"319962","messageId":"20170516203712.15921-1-avarab@gmail.com","threadId":"45981","inReplyTo":"20170516175906.hdwn4x5md7dj7fo3@kitenet.net","subject":"[PATCH] tests: add an optional test to test git-annex","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-16T20:37:12Z","receivedAt":"2017-05-16T20:37:28Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add an optional test to test git-annex. It's guarded by a new\nEXTERNAL_TESTS environment variable. Running this test takes me 10\nminutes.\n\nAs reported by Joey Hess in \"reversion in GIT_COMMON_DIR refs path\"[1]\ncommit f57f37e2e1 (\"files-backend: remove the use of git_path()\",\n2017-03-26) first released as part of the 2.13.0 broke git-annex's\ntest suite.\n\nThis could have been spotted by us before the release by optionally\nrunning the git-annex test suite as part of git itself. This optional\ntest does that. It currently fails due to the reported regression,\nbut, passes on the 2.12.0 release.\n\nThe git-annex revision to test can be specified with the\nGIT_TEST_GIT_ANNEX_REVISION environment variable. Joey has expressed\ninterest in testing development versions of git against git-annex[2],\nand can now test the latest revision with:\n\n    EXTERNAL_TESTS=1 GIT_TEST_GIT_ANNEX_REVISION='@{u}' ./t9950-git-annex.sh\n\nBy default the test finds the latest git-annex release tag and tests\nthat, since the primary purpose is to test regressions in git which\ncause git-annex to fail, not regressions in git-annex itself.\n\nThe t9* test namespace is currently full as documented in t/README. In\nlie of an empty t9X for \"external tools\" this change claims t995* for\nthat purpose.\n\n1. <20170516171028.5eagqr2sw5a2i77d@kitenet.net>\n   (https://public-inbox.org/git/20170516175906.hdwn4x5md7dj7fo3@kitenet.net/T/)\n2. http://git-annex.branchable.com/devblog/day_459__git_bug/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>\n---\n\nOn Tue, May 16, 2017 at 7:59 PM, Joey Hess <id@joeyh.name> wrote:\n> Ævar Arnfjörð Bjarmason wrote:\n>> On Tue, May 16, 2017 at 7:10 PM, Joey Hess <id@joeyh.name> wrote:\n>> I have no idea what this bug is about, but side-question: It looks\n>> like this is git-annex's own test suite that's failing with 2.13.0, is\n>> that right?\n>\n> Yes indeed.\n>\n>> It would be very nice to have a test in git itself to test with\n>> git-annex. I.e. some optional test that just pulls down the latest\n>> git-annex release & runs its tests against the git we're building.\n>>\n>> Thanks for annex b.t.w., I use it a lot.\n>\n> If the git devs are ok with this, I certianly would be happy if such\n> tests were run, at least occasionally, on the git side!\n\nI for one would run this test occasionally, and perhaps we could even\nrun it as part of Travis eventually (although there would be a *lot*\nof Haskell deps, on my box \"apt build-dep git-annex\" brought in 1/2 GB\nof packages).\n\nAs noted in the commit message, once this is part of git.git you can\neasily set an environment variable to test the bleeding edge of git\nagainst any arbitrary git-annex version.\n\n t/t9950-git-annex.sh | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 52 insertions(+)\n create mode 100755 t/t9950-git-annex.sh\n\ndiff --git a/t/t9950-git-annex.sh b/t/t9950-git-annex.sh\nnew file mode 100755\nindex 0000000000..2cbc1f4be3\n--- /dev/null\n+++ b/t/t9950-git-annex.sh\n@@ -0,0 +1,52 @@\n+#!/bin/sh\n+\n+test_description='the git-annex test suite'\n+. ./test-lib.sh\n+\n+if test -z \"$EXTERNAL_TESTS\"\n+then\n+\tskip_all='skipping tests of external tools. EXTERNAL_TESTS not defined'\n+\ttest_done\n+fi\n+\n+if test -n \"$NO_CURL\"\n+then\n+\tskip_all='skipping test, git built without http support'\n+\ttest_done\n+fi\n+\n+test_expect_success 'clone git-annex' '\n+\tgit clone https://git.joeyh.name/git/git-annex.git\n+'\n+\n+if test -n \"$GIT_TEST_GIT_ANNEX_REVISION\"\n+then\n+\ttest_expect_success \"plan to test git-annex $GIT_TEST_GIT_ANNEX_REVISION\" \"\n+\t\techo '$GIT_TEST_GIT_ANNEX_REVISION' >revision-to-test\n+\t\"\n+else\n+\ttest_expect_success \"plan to test git-annex's latest release tag\" '\n+\t\tgit -C git-annex tag --sort=version:refname -l \"[0-9]*.[0-9]*\" |\n+\t\t\ttail -n 1 >revision-to-test\n+\t'\n+fi\n+\n+test_expect_success 'checkout $(cat revision-to-test) for testing' '\n+\tgit -C git-annex checkout $(cat revision-to-test)\n+'\n+\n+test_expect_success 'build git-annex (if this fails, you are likely missing its Haskell dependencies' '\n+\t(\n+\t\tcd git-annex &&\n+\t\tmake\n+\t)\n+'\n+\n+test_expect_success 'test git-annex' '\n+\t(\n+\t\tcd git-annex &&\n+\t\tmake test\n+\t)\n+'\n+\n+test_done\n-- \n2.13.0.303.g4ebf302169\n\n"},{"id":"319969","messageId":"20170516221043.sgnieijh6bef6izm@kitenet.net","threadId":"45981","inReplyTo":"20170516203712.15921-1-avarab@gmail.com","subject":"Re: [PATCH] tests: add an optional test to test git-annex","fromName":"Joey Hess","fromEmail":"id@joeyh.name","sentAt":"2017-05-16T22:10:43Z","receivedAt":"2017-05-16T22:10:55Z","isPatch":true,"sender":{"key":"id@joeyh.name","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"Nice work.\n\nNote that you can export BUILDER=stack and git-annex will build with a\nknown good dependency stack, which can be more reliable/cross platform\nthan using apt to install its build dependencies. That needs\nhttps://docs.haskellstack.org/ installed. Also it currently needs\nGIT_TEST_GIT_ANNEX_REVISION=master since I improved git-annex's\nMakefile slightly.\n\n-- \nsee shy jo\n"},{"id":"319990","messageId":"xmqq7f1gyzep.fsf@gitster.mtv.corp.google.com","threadId":"45981","inReplyTo":"20170516203712.15921-1-avarab@gmail.com","subject":"Re: [PATCH] tests: add an optional test to test git-annex","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-17T02:45:18Z","receivedAt":"2017-05-17T02:45:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:\n\n> Add an optional test to test git-annex. It's guarded by a new\n> EXTERNAL_TESTS environment variable. Running this test takes me 10\n> minutes.\n\nWell, it is one thing to place git-annex under CI to make sure its\nlatest and greatest works together well with our latest and greatest\n(and it may be something we want to see happen), but driving its\ntests from our testsuite sounds like a tail wagging the dog, at\nleast to me.\n\nI do not mind at all to place the simple reproduction recipe Joey\nposted as a new test in our test suite, though.  That kind of test\nthat catches changes to externally visible behaviour surely belongs\nto our test suite.\n"},{"id":"320026","messageId":"CACBZZX4Jppr7ht7m444EjW4CDYX5CMvnxtStH4bF=A19TYKcZg@mail.gmail.com","threadId":"45981","inReplyTo":"xmqq7f1gyzep.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] tests: add an optional test to test git-annex","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-17T06:47:01Z","receivedAt":"2017-05-17T06:47:32Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Wed, May 17, 2017 at 4:45 AM, Junio C Hamano <gitster@pobox.com> wrote:\n> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:\n>\n>> Add an optional test to test git-annex. It's guarded by a new\n>> EXTERNAL_TESTS environment variable. Running this test takes me 10\n>> minutes.\n\n[Re-arranged your mail because it worked better with my reply]\n\n> I do not mind at all to place the simple reproduction recipe Joey\n> posted as a new test in our test suite, though.  That kind of test\n> that catches changes to externally visible behaviour surely belongs\n> to our test suite.\n\nThis is not a replacement for having an isolated test for the issue\nJoey noted. We should have a separate patch for that, but I did not\nhave time/interest in writing that up. This change is orthagonal to\nthat.\n\n> Well, it is one thing to place git-annex under CI to make sure its\n> latest and greatest works together well with our latest and greatest\n> (and it may be something we want to see happen), but driving its\n> tests from our testsuite sounds like a tail wagging the dog, at\n> least to me.\n\nTo me this is just a question of:\n\n* Is it the case that git-annex tests for a lot of edge cases we don't\ntest for: Yes, probably. As evidenced by them spotting this\nregression, and not us.\n\n* We can (and should) add a test for the specific breakage we caused\nin 2.13.0, but that's no replacement for other things annex may be\ncovering & we may be missing which'll catch future breakages.\n\n* It's a pretty established practice to test a library (git) along\nwith its consumers (e.g. annex) before a major release.\n\n* This allows us to do that at minimal cost. I think it makes sense to\nadd this and integration tests for other similar utilities if they're\nsimilarly easy to integrate.\n"},{"id":"320032","messageId":"xmqqbmqrwzu1.fsf@gitster.mtv.corp.google.com","threadId":"45981","inReplyTo":"CACBZZX4Jppr7ht7m444EjW4CDYX5CMvnxtStH4bF=A19TYKcZg@mail.gmail.com","subject":"Re: [PATCH] tests: add an optional test to test git-annex","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-17T10:19:02Z","receivedAt":"2017-05-17T10:19:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n\n>> Well, it is one thing to place git-annex under CI to make sure its\n>> latest and greatest works together well with our latest and greatest\n>> (and it may be something we want to see happen), but driving its\n>> tests from our testsuite sounds like a tail wagging the dog, at\n>> least to me.\n>\n> To me this is just a question of:\n>\n> * Is it the case that git-annex tests for a lot of edge cases we don't\n> test for: Yes, probably. As evidenced by them spotting this\n> regression, and not us.\n\nAnd I'd encourage them to keep doing so.\n\n> * We can (and should) add a test for the specific breakage we caused\n> in 2.13.0, but that's no replacement for other things annex may be\n> covering & we may be missing which'll catch future breakages.\n>\n> * It's a pretty established practice to test a library (git) along\n> with its consumers (e.g. annex) before a major release.\n\nI am not so sure about the division of labor.  What you are\nadvocating would work _ONLY_ if we test with a perfect & bug-free\nversion of the consumers.  If they are also a moving target, then\nI do not think it is worth it.  After all, we are *not* in the\nbusiness of testing these consumers.\n\nUnless I misunderstood you and you were saying that we freeze a\nversion, or a set of versions, of customer that is/are known to pass\ntheir own tests, and test the combination of that frozen version of\nthe customer with our daily development.  If that is the case, then\nI would agree that we are using their test to test us, not them.\nBut I somehow didn't get that impression, hence my reaction.\n"},{"id":"320100","messageId":"CACBZZX5BrASFemN2VviMjH-AqnxU6veLVmjRdn1iYuA9fgKQog@mail.gmail.com","threadId":"45981","inReplyTo":"xmqqbmqrwzu1.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] tests: add an optional test to test git-annex","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-17T19:33:15Z","receivedAt":"2017-05-17T19:33:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Wed, May 17, 2017 at 12:19 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n>\n>>> Well, it is one thing to place git-annex under CI to make sure its\n>>> latest and greatest works together well with our latest and greatest\n>>> (and it may be something we want to see happen), but driving its\n>>> tests from our testsuite sounds like a tail wagging the dog, at\n>>> least to me.\n>>\n>> To me this is just a question of:\n>>\n>> * Is it the case that git-annex tests for a lot of edge cases we don't\n>> test for: Yes, probably. As evidenced by them spotting this\n>> regression, and not us.\n>\n> And I'd encourage them to keep doing so.\n\nThe point of this patch is that we can do this more systematically and\nreliably, not have people discover this sort of thing after a major\nrelease.\n\nI.e. we can be pro-active about this instead of waiting for bug\nreports to roll in.\n\n>> * We can (and should) add a test for the specific breakage we caused\n>> in 2.13.0, but that's no replacement for other things annex may be\n>> covering & we may be missing which'll catch future breakages.\n>>\n>> * It's a pretty established practice to test a library (git) along\n>> with its consumers (e.g. annex) before a major release.\n>\n> I am not so sure about the division of labor.  What you are\n> advocating would work _ONLY_ if we test with a perfect & bug-free\n> version of the consumers.  If they are also a moving target, then\n> I do not think it is worth it.  After all, we are *not* in the\n> business of testing these consumers.\n>\n> Unless I misunderstood you and you were saying that we freeze a\n> version, or a set of versions, of customer that is/are known to pass\n> their own tests, and test the combination of that frozen version of\n> the customer with our daily development.  If that is the case, then\n> I would agree that we are using their test to test us, not them.\n> But I somehow didn't get that impression, hence my reaction.\n\nThe test I'm adding tests the release version of git-annex, so I think\nin practice we don't have to worry about random changes of theirs\nproducing false positives for us.\n\nThe utility of this test is that sometime close to release someone\n(e.g. me) can run it, if it fails let's see if it fails on the last\nrelease version of ours, if so it's probably upstream breakage, or\nlike with the 2.13.0 release if it's OK on 2.12.0 it's our bug.\n\nIt'll never trip some random tester up since you need to explicitly\nopt-in via EXTERNAL_TESTS=1, so honestly I'm a bit puzzled by these\nobjections. This incurs no burden on either devs, packagers or users,\nand would have demonstrably detected an issue we'd rather have wanted\nto know about pre-release than post-release as is the case now.\n"},{"id":"320112","messageId":"CAGZ79kYX8ct4GKDbZxGJmR5kkVrs4zjnaKOaD8Dm8rKx8aPA+Q@mail.gmail.com","threadId":"45981","inReplyTo":"CACBZZX5BrASFemN2VviMjH-AqnxU6veLVmjRdn1iYuA9fgKQog@mail.gmail.com","subject":"Re: [PATCH] tests: add an optional test to test git-annex","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-05-17T23:03:48Z","receivedAt":"2017-05-17T23:03:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, May 17, 2017 at 12:33 PM, Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n> On Wed, May 17, 2017 at 12:19 PM, Junio C Hamano <gitster@pobox.com> wrote:\n>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n>>\n>>>> Well, it is one thing to place git-annex under CI to make sure its\n>>>> latest and greatest works together well with our latest and greatest\n>>>> (and it may be something we want to see happen), but driving its\n>>>> tests from our testsuite sounds like a tail wagging the dog, at\n>>>> least to me.\n>>>\n>>> To me this is just a question of:\n>>>\n>>> * Is it the case that git-annex tests for a lot of edge cases we don't\n>>> test for: Yes, probably. As evidenced by them spotting this\n>>> regression, and not us.\n>>\n>> And I'd encourage them to keep doing so.\n>\n> The point of this patch is that we can do this more systematically and\n> reliably, not have people discover this sort of thing after a major\n> release.\n>\n> I.e. we can be pro-active about this instead of waiting for bug\n> reports to roll in.\n\nWe can be pro-active without this patch, too. ;)\nI guess this makes it just easier for someone out of the Git community to\nbe proactive in searching for defects.\n\n>\n> The utility of this test is that sometime close to release someone\n> (e.g. me) can run it, if it fails let's see if it fails on the last\n> release version of ours, if so it's probably upstream breakage, or\n> like with the 2.13.0 release if it's OK on 2.12.0 it's our bug.\n>\n> It'll never trip some random tester up since you need to explicitly\n> opt-in via EXTERNAL_TESTS=1, so honestly I'm a bit puzzled by these\n> objections. This incurs no burden on either devs, packagers or users,\n> and would have demonstrably detected an issue we'd rather have wanted\n> to know about pre-release than post-release as is the case now.\n\nI think this patch could spark a discussion what we expect from our test suite.\nIs it a contract that we promise, as in \"Here are some blessed workflows,\nour users should see and imitate\"?\n\nThis patch sort of feels like (a) we let others dictate the contract to us\nand (b) we choose the easy way out as we do not write enough tests on\nour own, so we force downstream to help us out.\n\nI guess it is appropriate to compare this level of testing to other external\ntests, such as cppcheck, travis, coverity. For some of them (travis) we\ncarry some code with us (.travis), not so for others (coverity would benefit\nfrom [1] that I carry outside of Junios tree). Admittedly these external tools\nare all focused on testing, not on building on top of Git, which brings\nin other expectations from these tools.\n\n[1] https://github.com/stefanbeller/git/commit/0781fbafb9dd2a995ba62a9af5f7581e3cf05359\n\n---\nAs mentioned in the other thread[2], the idea was floated to have this test or\nthe downstream project as a submodule, and if you are interested in the test,\nthen you would obtain the submodule and run the tests (in there?)\nThat would have the benefit of less 'uninteresting' glue logic for those who\nare not interested in these tests as gitlinks may be easier to ignore\nfor a developer\nthan code. (Less data to fetch, git-grep also doesn't yield results\nfor the uninteresting\nparts as these submodules would be uninitialized).\n(The more I talk about this \"downstream tests in submodules\"\nidea, the less convinced I am)\n\n[2] https://public-inbox.org/git/20170517113824.31700-1-avarab@gmail.com/\n"},{"id":"320113","messageId":"20170517235626.GB185461@google.com","threadId":"45981","inReplyTo":"xmqqbmqrwzu1.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] tests: add an optional test to test git-annex","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-05-17T23:56:26Z","receivedAt":"2017-05-17T23:56:34Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 05/17, Junio C Hamano wrote:\n> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n> \n> >> Well, it is one thing to place git-annex under CI to make sure its\n> >> latest and greatest works together well with our latest and greatest\n> >> (and it may be something we want to see happen), but driving its\n> >> tests from our testsuite sounds like a tail wagging the dog, at\n> >> least to me.\n> >\n> > To me this is just a question of:\n> >\n> > * Is it the case that git-annex tests for a lot of edge cases we don't\n> > test for: Yes, probably. As evidenced by them spotting this\n> > regression, and not us.\n> \n> And I'd encourage them to keep doing so.\n> \n> > * We can (and should) add a test for the specific breakage we caused\n> > in 2.13.0, but that's no replacement for other things annex may be\n> > covering & we may be missing which'll catch future breakages.\n> >\n> > * It's a pretty established practice to test a library (git) along\n> > with its consumers (e.g. annex) before a major release.\n> \n> I am not so sure about the division of labor.  What you are\n> advocating would work _ONLY_ if we test with a perfect & bug-free\n> version of the consumers.  If they are also a moving target, then\n> I do not think it is worth it.  After all, we are *not* in the\n> business of testing these consumers.\n\nI agree with this. It makes no sense to test consumers of git, its\ndownstream's job to do that.  Though I do think that its perfectly\nreasonable to test that our API works as advertised such that consumer's\ncan rely on git.\n\n> \n> Unless I misunderstood you and you were saying that we freeze a\n> version, or a set of versions, of customer that is/are known to pass\n> their own tests, and test the combination of that frozen version of\n> the customer with our daily development.  If that is the case, then\n> I would agree that we are using their test to test us, not them.\n> But I somehow didn't get that impression, hence my reaction.\n\n-- \nBrandon Williams\n"},{"id":"320114","messageId":"20170517235947.GC185461@google.com","threadId":"45981","inReplyTo":"CAGZ79kYX8ct4GKDbZxGJmR5kkVrs4zjnaKOaD8Dm8rKx8aPA+Q@mail.gmail.com","subject":"Re: [PATCH] tests: add an optional test to test git-annex","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-05-17T23:59:47Z","receivedAt":"2017-05-17T23:59:55Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 05/17, Stefan Beller wrote:\n> On Wed, May 17, 2017 at 12:33 PM, Ævar Arnfjörð Bjarmason\n> <avarab@gmail.com> wrote:\n> > On Wed, May 17, 2017 at 12:19 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> >> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:\n> >>\n> >>>> Well, it is one thing to place git-annex under CI to make sure its\n> >>>> latest and greatest works together well with our latest and greatest\n> >>>> (and it may be something we want to see happen), but driving its\n> >>>> tests from our testsuite sounds like a tail wagging the dog, at\n> >>>> least to me.\n> >>>\n> >>> To me this is just a question of:\n> >>>\n> >>> * Is it the case that git-annex tests for a lot of edge cases we don't\n> >>> test for: Yes, probably. As evidenced by them spotting this\n> >>> regression, and not us.\n> >>\n> >> And I'd encourage them to keep doing so.\n> >\n> > The point of this patch is that we can do this more systematically and\n> > reliably, not have people discover this sort of thing after a major\n> > release.\n> >\n> > I.e. we can be pro-active about this instead of waiting for bug\n> > reports to roll in.\n> \n> We can be pro-active without this patch, too. ;)\n> I guess this makes it just easier for someone out of the Git community to\n> be proactive in searching for defects.\n> \n> >\n> > The utility of this test is that sometime close to release someone\n> > (e.g. me) can run it, if it fails let's see if it fails on the last\n> > release version of ours, if so it's probably upstream breakage, or\n> > like with the 2.13.0 release if it's OK on 2.12.0 it's our bug.\n> >\n> > It'll never trip some random tester up since you need to explicitly\n> > opt-in via EXTERNAL_TESTS=1, so honestly I'm a bit puzzled by these\n> > objections. This incurs no burden on either devs, packagers or users,\n> > and would have demonstrably detected an issue we'd rather have wanted\n> > to know about pre-release than post-release as is the case now.\n> \n> I think this patch could spark a discussion what we expect from our test suite.\n> Is it a contract that we promise, as in \"Here are some blessed workflows,\n> our users should see and imitate\"?\n> \n> This patch sort of feels like (a) we let others dictate the contract to us\n> and (b) we choose the easy way out as we do not write enough tests on\n> our own, so we force downstream to help us out.\n> \n> I guess it is appropriate to compare this level of testing to other external\n> tests, such as cppcheck, travis, coverity. For some of them (travis) we\n> carry some code with us (.travis), not so for others (coverity would benefit\n> from [1] that I carry outside of Junios tree). Admittedly these external tools\n> are all focused on testing, not on building on top of Git, which brings\n> in other expectations from these tools.\n> \n> [1] https://github.com/stefanbeller/git/commit/0781fbafb9dd2a995ba62a9af5f7581e3cf05359\n> \n> ---\n> As mentioned in the other thread[2], the idea was floated to have this test or\n> the downstream project as a submodule, and if you are interested in the test,\n> then you would obtain the submodule and run the tests (in there?)\n\nDespite working on improving submodules, I'm still very skeptical of\nadding submodules to git.git, at least at this point in time.  I just\nfeel like the submodule experience still has a lot of work before we\nbegin using it...Though My feeling may be completely unfounded.\n\n> That would have the benefit of less 'uninteresting' glue logic for those who\n> are not interested in these tests as gitlinks may be easier to ignore\n> for a developer\n> than code. (Less data to fetch, git-grep also doesn't yield results\n> for the uninteresting\n> parts as these submodules would be uninitialized).\n> (The more I talk about this \"downstream tests in submodules\"\n> idea, the less convinced I am)\n> \n> [2] https://public-inbox.org/git/20170517113824.31700-1-avarab@gmail.com/\n\n-- \nBrandon Williams\n"},{"id":"320274","messageId":"20170519143727.edi4ni7v5pywm7dk@kitenet.net","threadId":"45981","inReplyTo":"20170516171028.5eagqr2sw5a2i77d@kitenet.net","subject":"Re: reversion in GIT_COMMON_DIR refs path","fromName":"Joey Hess","fromEmail":"id@joeyh.name","sentAt":"2017-05-19T14:37:27Z","receivedAt":"2017-05-19T14:37:41Z","isPatch":false,"sender":{"key":"id@joeyh.name","avatar":"https://avatars.githubusercontent.com/u/16392?v=4"},"body":"Joey Hess wrote:\n> Bisecting this test suite failure\n> https://git-annex.branchable.com/git-annex_in_nixpkgs_fails_with_git-2.13.0/\n> I landed on commit f57f37e2e1bf11ab4cdfd221ad47e961ba9353a0 to git.\n> \n> It seems that changed resolving refs paths when GIT_DIR and GIT_COMMON_DIR\n> are both set. While before refs were looked for in GIT_COMMON_DIR,\n> now they're not.\n\nIn case there's any doubt about whether this is a reversion or an\nintentional change, see gitrepository-layout(5):\n\n       refs\n           References are stored in subdirectories of this directory. The git\n           prune command knows to preserve objects reachable from refs found\n           in this directory and its subdirectories. This directory is ignored\n           if $GIT_COMMON_DIR is set and \"$GIT_COMMON_DIR/refs\" will be used\n           instead.\n\nSo the documented behavior is broken.\n\n-- \nsee shy jo\n"},{"id":"320401","messageId":"CACsJy8C4EZnB2PkJw5t7c007vRmb8DDKk41p2azvjOnJNO0n7Q@mail.gmail.com","threadId":"45981","inReplyTo":"20170519143727.edi4ni7v5pywm7dk@kitenet.net","subject":"Re: reversion in GIT_COMMON_DIR refs path","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2017-05-22T11:11:21Z","receivedAt":"2017-05-22T11:12:02Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, May 19, 2017 at 9:37 PM, Joey Hess <id@joeyh.name> wrote:\n> Joey Hess wrote:\n>> Bisecting this test suite failure\n>> https://git-annex.branchable.com/git-annex_in_nixpkgs_fails_with_git-2.13.0/\n>> I landed on commit f57f37e2e1bf11ab4cdfd221ad47e961ba9353a0 to git.\n>>\n>> It seems that changed resolving refs paths when GIT_DIR and GIT_COMMON_DIR\n>> are both set. While before refs were looked for in GIT_COMMON_DIR,\n>> now they're not.\n>\n> In case there's any doubt about whether this is a reversion or an\n> intentional change, see gitrepository-layout(5):\n>\n>        refs\n>            References are stored in subdirectories of this directory. The git\n>            prune command knows to preserve objects reachable from refs found\n>            in this directory and its subdirectories. This directory is ignored\n>            if $GIT_COMMON_DIR is set and \"$GIT_COMMON_DIR/refs\" will be used\n>            instead.\n>\n> So the documented behavior is broken.\n\nIt's a gray area. When I wrote that I think I forgot about\nper-worktree refs (refs/bisect/*) so \"This directory is ignored\" is\nnot completely true. The final line (probably won't help you much) is\n\"per-repo refs must be read from $GIT_COMMON_DIR/refs, per-worktree\nfrom $GIT_DIR\". The fact that we looked per-repo (like master) in\n$GIT_DIR is probably an unwanted side effect.\n-- \nDuy\n"}]}