{"thread":{"id":"51686","subject":"Fully peel tags via for-each-ref?","startedAt":"2019-08-19T23:22:19Z","lastAt":"2019-08-22T04:24:47Z","messageCount":6,"participants":["Bryan Turner","Junio C Hamano","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"380747","messageId":"CAGyf7-GBx3FSCQTipmkNBtnwMANg5A0FXiiPc2az0NiRLRT+xg@mail.gmail.com","threadId":"51686","inReplyTo":null,"subject":"Fully peel tags via for-each-ref?","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2019-08-19T23:22:05Z","receivedAt":"2019-08-19T23:22:19Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"When running \"git for-each-ref\", it's possible to use %(*objectname)\nto get the SHA of the object a tag is tagging, and %(*objecttype) to\nget the type. However, in the case of an annotated tag of an annotated\ntag, this still isn't \"fully\" peeled--%(*objectname) is the tagged\ntag's SHA.\n\nIs there any way, with \"git for-each-ref\", to output the \"fully\"\npeeled SHA of a tag's ultimate target, regardless of how many layers\nmust be traversed?\n\nBest regards,\nBryan Turner\n"},{"id":"380772","messageId":"xmqq8srowfkv.fsf@gitster-ct.c.googlers.com","threadId":"51686","inReplyTo":"CAGyf7-GBx3FSCQTipmkNBtnwMANg5A0FXiiPc2az0NiRLRT+xg@mail.gmail.com","subject":"Re: Fully peel tags via for-each-ref?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-20T05:50:24Z","receivedAt":"2019-08-20T05:50:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Bryan Turner <bturner@atlassian.com> writes:\n\n> Is there any way, with \"git for-each-ref\", to output the \"fully\"\n> peeled SHA of a tag's ultimate target, regardless of how many layers\n> must be traversed?\n\nI do not think I wrote it to allow different degree of peeling, not\nbecause I wanted to explicitly forbid a use case for tags that tag\nanother tag, but simply because I didn't think of anybody using it\nand didn't see need to support such tags.\n\nIf %(*<stuff>) does not peel fully (I do not recall what I did\noffhand), because all other things in Git (like $X~0, $X^{tree},\netc.) fully peel the outer object until they get to what they want,\nit may even be OK to declare it a bug and \"fix\" the notation to\nfully peel tags.  I dunno.\n"},{"id":"380866","messageId":"CAGyf7-F_89u-913P0znH_3NCzNZNarb3u_3Vpwt5-QMiFCK+Tg@mail.gmail.com","threadId":"51686","inReplyTo":"xmqq8srowfkv.fsf@gitster-ct.c.googlers.com","subject":"Re: Fully peel tags via for-each-ref?","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2019-08-20T21:39:34Z","receivedAt":"2019-08-20T21:39:48Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"On Mon, Aug 19, 2019 at 10:50 PM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Bryan Turner <bturner@atlassian.com> writes:\n>\n> > Is there any way, with \"git for-each-ref\", to output the \"fully\"\n> > peeled SHA of a tag's ultimate target, regardless of how many layers\n> > must be traversed?\n>\n>\n> If %(*<stuff>) does not peel fully (I do not recall what I did\n> offhand), because all other things in Git (like $X~0, $X^{tree},\n> etc.) fully peel the outer object until they get to what they want,\n> it may even be OK to declare it a bug and \"fix\" the notation to\n> fully peel tags.  I dunno.\n\n%(*<stuff>) definitely doesn't fully peel.\n\n$ git for-each-ref --format=\"%(refname) %(objectname) ->\n%(*objectname) (%(*objecttype))\" refs/tags/\nrefs/tags/backdated_annotated_tag\n80aa5be5ecf39660f798858482254f7a2ae9110e ->\n57150c54c38d6570b2fd5e6d6cfc19476de44e84 (commit)\nrefs/tags/retagged_signed_tag 9b4e781dea0769888fe270e06ad76675f73851b1\n-> 12ebe2a58367347cd39f19f5a72f3cbec7b8f9a9 (tag)\nrefs/tags/signed_tag 12ebe2a58367347cd39f19f5a72f3cbec7b8f9a9 ->\n0a943a29376f2336b78312d99e65da17048951db (commit)\n\n\"retagged_signed_tag\" is tagging \"signed_tag\", and you can see that\n%(*objectname) for it is \"signed_tag\"'s hash and %(*objecttype) is\ntag.\n\n$ git rev-parse retagged_signed_tag~0\n0a943a29376f2336b78312d99e65da17048951db\n\nI don't know that this use case is common enough for it to warrant\n\"fixing\". I'm not even sure most would call it a \"bug\"; I hadn't\nconsidered it that way, at least. I was just interested in whether\nthere might be an out-of-the-box approach I wasn't aware of for\ngetting fully peeled results.\n\nThanks for taking the time to answer, and to provide a bit of history!\n\nBest regards,\nBryan Turner\n"},{"id":"380917","messageId":"20190821230035.GA26107@sigill.intra.peff.net","threadId":"51686","inReplyTo":"xmqq8srowfkv.fsf@gitster-ct.c.googlers.com","subject":"Re: Fully peel tags via for-each-ref?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-21T23:00:36Z","receivedAt":"2019-08-21T23:00:40Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Aug 19, 2019 at 10:50:24PM -0700, Junio C Hamano wrote:\n\n> Bryan Turner <bturner@atlassian.com> writes:\n> \n> > Is there any way, with \"git for-each-ref\", to output the \"fully\"\n> > peeled SHA of a tag's ultimate target, regardless of how many layers\n> > must be traversed?\n> \n> I do not think I wrote it to allow different degree of peeling, not\n> because I wanted to explicitly forbid a use case for tags that tag\n> another tag, but simply because I didn't think of anybody using it\n> and didn't see need to support such tags.\n> \n> If %(*<stuff>) does not peel fully (I do not recall what I did\n> offhand), because all other things in Git (like $X~0, $X^{tree},\n> etc.) fully peel the outer object until they get to what they want,\n> it may even be OK to declare it a bug and \"fix\" the notation to\n> fully peel tags.  I dunno.\n\nYeah, my first thought on reading Bryan's email was that this is so\ninconsistent with the rest of Git as to be considered a bug.\n\nThere's this gem in ref-filter.c, which blames back to your 9f613ddd21\n(Add git-for-each-ref: helper for language bindings, 2006-09-15):\n\n          /*\n           * NEEDSWORK: This derefs tag only once, which\n           * is good to deal with chains of trust, but\n           * is not consistent with what deref_tag() does\n           * which peels the onion to the core.\n           */\n          return get_object(ref, 1, &obj, &oi_deref, err);\n\nWhich isn't to say it isn't useful to be able to do a single-layer peel,\nbut I can't think of another part of the system which does so (unless\nyou've asked to peel to a specific type, of course). I'm thinking\nespecially of the way upload-pack does its advertisement, with two\ninteresting implications:\n\n  1. We store full peels in the packed-refs file[1], so we can show them\n     in the advertisement without having to access the object. Doing:\n\n       git upload-pack . </dev/null >/dev/null\n\n     is much cheaper than:\n\n       git for-each-ref \\\n         --format='%(objectname) %(refname)%0a%(*objectname) %(refname)^{}' \\\n\t >/dev/null\n\n  2. When we switched for_each_alternate_ref to using peeled values, we\n     stopped advertising peeled values for .push haves entirely. There's\n     some discussion in a10a17877b (for_each_alternate_ref: replace\n     transport code with for-each-ref, 2017-02-08), but if we _did_ want\n     to show them, we'd probably want the fully peeled value to match\n     the rest of the advertisement.\n\nGiven that it's not how the rest of Git works, I'd be surprised for\nanybody to be relying on the shallow-peeling behavior of for-each-ref.\nBut if we did want to retain it, it might be worth adding a separate\nsyntax for a full peel. \"%(**objectname)\" or something?\n\n-Peff\n\n[1] Storing peeled values in the packed-refs file is itself a pretty\n    horrible hack. It's really a property of the object itself, and\n    we're just piggy-backing on the ref storage to make this cached\n    metadata easily accessible. It would be more appropriate in an\n    object metadata storage format similar to the commit-graph file.\n    And then of course we could store both full or partial peels,\n    depending on the format.\n"},{"id":"380919","messageId":"xmqqmug1rj7y.fsf@gitster-ct.c.googlers.com","threadId":"51686","inReplyTo":"20190821230035.GA26107@sigill.intra.peff.net","subject":"Re: Fully peel tags via for-each-ref?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-08-22T03:07:29Z","receivedAt":"2019-08-22T03:07:40Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> There's this gem in ref-filter.c, which blames back to your 9f613ddd21\n> (Add git-for-each-ref: helper for language bindings, 2006-09-15):\n>\n>           /*\n>            * NEEDSWORK: This derefs tag only once, which\n>            * is good to deal with chains of trust, but\n>            * is not consistent with what deref_tag() does\n>            * which peels the onion to the core.\n>            */\n>           return get_object(ref, 1, &obj, &oi_deref, err);\n>\n> Which isn't to say it isn't useful to be able to do a single-layer peel,\n> but I can't think of another part of the system which does so (unless\n> you've asked to peel to a specific type, of course).\n\nQuite honestly, I think the \"only once\" behaviour outlived its\nusefulness, without other \"features\" that may help make it more\nuseful.  To help a script that wants to do \"chains of trust\", it may\nfirst appear to be useful to peel only one layer, revealing that the\ntagged object is another tag, and that was the thinking behind the\nNEEDSWORK comment.\n\nBut after we learn that a ref \"refs/tags/foo\" points at a tag object\nthat points at another tag object, what can the script do?  It\ncannot feed the other tag found that way back into --format=%(*<thing>)\nmachinery of for-each-ref, as the command and its variants, the \"--list\"\nmode of \"branch\" and \"tag\" commands, only work on the object at the\ntip of refs.  The script must manually peel the tag one layer at a time.\n\nAnd that \"manually\" has to be really manual.  No ^{<type>} suffix\nallows scripts to peel just one layer, so inside a loop, the script\nhas to say \"cat-file tag <object>\" and parse the \"object\" header\nfrom the output.\n\nThe only thing that gets affected if we changed %(*<thing>) to fully\npeel tags is a tag that points at another tag, and the traditional\nbehaviour to peel only one layer, while it _might_ have been done as\na good first step to add more support for chain of trust, is not all\nthat useful for such a tag, I am not sure if the current behaviour\nis defensible.\n"},{"id":"380923","messageId":"20190822042445.GB4347@sigill.intra.peff.net","threadId":"51686","inReplyTo":"xmqqmug1rj7y.fsf@gitster-ct.c.googlers.com","subject":"Re: Fully peel tags via for-each-ref?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-08-22T04:24:45Z","receivedAt":"2019-08-22T04:24:47Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Aug 21, 2019 at 08:07:29PM -0700, Junio C Hamano wrote:\n\n> But after we learn that a ref \"refs/tags/foo\" points at a tag object\n> that points at another tag object, what can the script do?  It\n> cannot feed the other tag found that way back into --format=%(*<thing>)\n> machinery of for-each-ref, as the command and its variants, the \"--list\"\n> mode of \"branch\" and \"tag\" commands, only work on the object at the\n> tip of refs.  The script must manually peel the tag one layer at a time.\n> \n> And that \"manually\" has to be really manual.  No ^{<type>} suffix\n> allows scripts to peel just one layer, so inside a loop, the script\n> has to say \"cat-file tag <object>\" and parse the \"object\" header\n> from the output.\n> \n> The only thing that gets affected if we changed %(*<thing>) to fully\n> peel tags is a tag that points at another tag, and the traditional\n> behaviour to peel only one layer, while it _might_ have been done as\n> a good first step to add more support for chain of trust, is not all\n> that useful for such a tag, I am not sure if the current behaviour\n> is defensible.\n\nYeah, well said, and I think this is the heart of the matter. There\nisn't good tooling for this case, but even if there was, for-each-ref is\ndefinitely not the place for it.\n\nI took a brief look at making a patch, and it's unfortunately\nnon-trivial because of the way the deref logic in ref-filter is\nimplemented. So it may not be worth digging too far into for now if\nnobody cares much either way.\n\nI still have dreams of writing a more efficient and more reusable\nversion of ref-filter, and it would probably make sense to use\npeel_ref() as part of that rewrite.\n\n-Peff\n"}]}